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This  report  provides  a  review  of  literature  on  computer  menu  interface  design  and  related  performance  factors. 
Implications  for  the  design  of  the  user  system  interface  for  aircraft  simulator  instructional  support  systems  are 
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pages  affects  the  search  process  and  the  pattern  of  errors  that  will  result  is  evaluated.  The  literature  on  theoretical 
and  empirical  work  suggests  two  additional  factors  that  are  recommended  for  inclusion  into  the  search  model: 
(a)  user-perceived  relationships  among  target  items  sought  and  menu  alternatives  available  for  selection,  and  (b) 
the  probability  of  an  omission  situation  where  the  target  item  is  not  subsumed  under  any  of  the  alternatives  available 
for  selection  An  experiment  was  conducted  *0  test  the  effect  that  all  three  of  these  factors  have  on  menu  task 
performance  Results  showed  that  all  three  factors  significantly  influenced  menu  search  and  response  accuracy 
A  two-criterion  menu  model  was  proposed  as  a  means  to  explicate  the  performance  results  of  the  menu 
experiment. 
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COMPUTER  MENU  TASK  PERFORMANCE  MODEL  DEVELOPMENT 


I.  INTRODUCTION 
Statement  of  the  Problem 


Ground-based  aircrew  training  devices  (ATDs)  may  be  conceptualized  as  consisting  of  two 
main  components:  the  simulation  system  and  the  instructional  support  system  (ISS).  The 
simulation  component  encompasses  all  that  is  required  to  replicate  the  aircraft  and  the  flight 
environment.  It  includes  all  cockpit,  visual  and  database  systems,  along  with  the  computer 
hardware  and  software  required  to  support  them.  Most  behavioral  research  conducted  on  ATDs 
has  focused  on  issues  concerning  the  required  fidelity  (stimulus  cues)  of  the  simulation. 

The  other  component,  the  ISS,  provides  all  interface  controls  and  displays  required  to  use 
the  simulator  as  a  training  device.  The  ISS  also  can  provide  briefing  and  debriefing  capabilities, 
as  well  as  a  variety  of  other  instructional  support  functions.  With  advances  in  visual  simulation, 
computer,  and  networking  technologies,  the  training  capabilities  of  ATDs  have  increased  dramatically. 
These  advances  have  increased  the  complexity  of  ISS  functions  and,  as  a  result,  the  communication 
between  the  user  and  the  ISS  has  become  more  difficult. 

The  negative  effect  that  poorly  designed  ISS  interfaces  have  had  on  the  use  of  ATDs  has 
been  well  documented.  The  United  States  Air  Force  has  completed  a  series  of  research  efforts 
to  assess  the  effectiveness  of  operational  ATDs  in  the  Air  Force  inventory  (Polzella,  Hubbard, 
Brown,  &  McLean,  1987;  Semple,  Cotton,  &  Sullivan,  1981).  These  studies  examined  the  utilization 
of  these  training  devices,  the  use  and  value  of  ISS  features,  and  the  overall  acceptance  of  the 
training  systems  by  the  students  and  instructors  who  used  them.  The  results  revealed  that  many 
ATDs  were  not  being  used  as  expected,  that  user  acceptance  was  sometimes  low,  and  that 
many  trainer  capabilities  were  not  being  utilized.  These  studies  suggested  that  much  of  the 
problem  can  be  attributed  to  the  fact  that  information  displays  and  controls  required  to  employ 
ATD  capabilities  effectively  are  often  either  not  available  or  configured  in  a  manner  that  is  difficult 
to  use.  These  findings  provided  the  genesis  for  the  work  described  in  this  report. 


User/Computer  Communication  Techniques 


Many  practical  alternatives  exist  for  communication  between  the  computer  and  its  user.  All 
have  certain  advantages  and  disadvantages,  which  usually  are  dependent  on  the  characteristics 
of  the  typical  user  of  the  system  (Hauptman  &  Green,  1983).  Menu  design  has  been  found 
especially  effective  for  systems  that  have  new,  infrequent  or  untrained  users,  because  the  user 
need  not  memorize  and  recall  commands  but  merely  recognize  them.  Thus,  effective  menu 
design  has  become  an  area  of  concern  for  designers  of  simulator  ISSs  where  the  end  user  is 
typically  an  instructor  or  a  student  pilot  who  uses  the  system  on  a  recurring  but  infrequent 
basis.  Via  menus,  whenever  the  user  must  control  computer  actions  the  program  displays  a 
set  of  options  describing  all  the  alternatives  available  at  that  point.  Th-e  user  selects  one  of 
the  alternatives  by  keying-in  a  response  that  activates  the  option  desired.  The  program  then 
branches  to  the  subroutine  that  corresponds  to  the  user's  choice  and  displays  a  new  menu  of 
available  options.  This  process  is  repeated  until  the  user  finds  and  selects  the  end-level  option 
sought. 

The  primary  advantage  of  a  menu-driven  system  is  that  the  user  has  only  to  know  and 
understand  the  current  options  available  at  any  particular  point  of  execution.  Little  formal  training 
is  required  because  this  technique  relies  mainly  on  the  operator’s  general  subject-matter  expertise 
and  requires  only  passive  responses  to  computer  prompts.  The  main  drawbacks  associated 
with  menu-driven  systems  stem  from  the  need  to  organize  all  available  command  options  into 
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a  particular  menu  structure  within  the  program.  Thus,  the  creator  of  the  menu  not  only  must 
know  all  the  possible  or  desirable  options  to  include,  but  also  must  organize  this  material  into 
a  menu  configuration  that  accommodates  a  wide  variety  of  users. 

Even  though  the  utility  of  menu-driven  systems  to  ISS  applications  has  been  well  accepted, 
a  need  exists  for  expanded  understanding  of  the  performance  effects  of  menu  design  factors. 
-Such  information  could  be  used  to  structure  menus  so  as  to  optimize  user  performance.  Menu 
design  factors  which  have  been  found  to  impact  user  performance  include  breadth  (number  of 
items  displayed  on  each  menu  level),  depth  (number  of  levels  within  a  menu  structure),  and 
item  organization.  The  objective  of  the  present  investigation  was  to  quantify  and  model  the 
effects  these  factors  have  on  user  performance  at  rudimentary  levels  of  task  familiarity.  The 
ultimate  goal  was  to  build  a  theoretical  foundation  which  can  be  applied  specifically  to  the 
design  of  menus  for  ISSs. 


II.  BACKGROUND  LITERATURE 

In  very  simple  applications  where  the  number  of  functions  performed  by  a  system  are  relatively 
few,  menu  commands  can  usually  be  arranged  on  a  single  page  from  which  the  user  selects 
the  command  desired.  In  more  complicated  applications  where  the  system  is  capable  of 
performing  numerous  functions,  a  more  sophisticated  organization  of  commands  is  required. 
Large  menus  typically  are  organized  into  hierarchies  with  varying  levels  of  menu  pages  and 
options  These  hierarchies  can  be  arranged  with  many  items  on  a  menu  and  a  minimum  number 
of  sequential  menu  levels  (breadth),  with  few  items  on  each  menu  and  several  levels  (depth), 
or  with  some  intermediate  level  of  breadth  and  depth.  Recommendations  concerning  the  use 
of  breadth  versus  depth  in  menu  construction  have  been  based  on  quantitative  and  empirical 
studies,  as  well  as  implications  derived  from  cognitive  theories.  What  follows  is  a  general  review 
of  experiments  and  theories  that  are  pertinent  to  the  design  of  computer  menu  interfaces. 


Modeling  Menu  Requirements 


Only  a  limited  number  of  quantitative  approaches  for  resolving  menu  breadth/depth  issues 
have  been  investigated.  Lee  and  MacGregor  (1985)  described  a  mathematical  model  for  calculating 
optimal  menu  breadth  as  a  function  of  human  and  computer  factors  of  search  strategy,  scanning 
time,  keypress  time  and  computer  response  time.  The  model  was  based  on  the  assumption 
that  the  usefulness  of  a  menu  depends  on  the  amount  of  time  required  to  retrieve  information 
associated  with  its  use.  Thus,  the  model  calculates  optimal  breadth  by  minimizing  the  total  time 
required  of  the  user  to  navigate  through  a  menu  and  select  the  target  (i.e.,  end-level  item) 
desired.  Model  solutions  are  dependent  upon  the  search  strategy  to  be  employed  by  the  user. 
Possible  search  strategies  include:  self-terminating,  wherein  the  user  terminates  the  search  process 
as  soon  as  the  selected  alternative  is  encountered  in  the  menu;  exhaustive,  wherein  the  user 
reads  all  alternatives  in  the  menu  before  responding;  and  redundant,  wherein  the  user  examines 
several  or  all  alternatives  more  than  once  before  responding. 

Lee  and  MacGregor  (1985)  used  this  model  to  calculate  the  optimum  number  of  alternatives 
for  reading  rates,  keypress  times  and  computer  response  times  typical  of  menu-based  videotex 
information  retrieval  systems.  Videotex  is  an  interactive  system  designed  to  present  textual  and 
graphic  information  at  the  command  of  the  user  (Tydeman,  Lipinski,  Adler,  Nyhan,  &  Swimpfer, 
1982).  The  user  typically  employs  a  keypad  or  keyboard  and  a  computer  terminal  to  interact 
with  a  central  computer  via  telephone  lines  or  hardwire  connection.  Videotex  systems  involve 
menus  which  may  consist  of  thousands  of  menu  pages  with  menu  alternatives  that  are  frequently 
phrases  or  lists.  For  the  range  of  values  examined,  the  authors  concluded  that  4  to  8  alternatives 
per  page  were  optimal  to  minimize  user  search  time. 
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MacGregor,  Lee,  and  Lam  (1986)  refined  this  modei  and  examined  those  factors  which  directiy 
impact  upon  the  search  processes  empioyed  by  users.  They  pointed  out  that  for  conditions 
where  computer  response  time  Is  long  and  the  decision  time  shon,  the  optimal  breadth  can 
excood  the  '4  to  8"  range  that  Lee  and  MacGregor  (1985)  recommended.  Such  conditions  are 
typical  of  command-type  menu  applications.  By  comparison  to  videotex  systems,  command-type 
menus  involve  a  relatively  limited  set  of  alternatives,  where  learning  of  items  and  their  location 
in  a  menu  can  occur  quite  rapidly.  Command  items  themselves  are  frequently  single  words 
requiring  littie  search  and  processing  time.  MacGregor  et  al.  (1986)  argued  that  the  greater 
menu  breadth  required  by  command  menu  applications  is  predicted  by  the  search  model  and 
that  the  "4  to  8"  range  previously  suggested  was  based  on  parameter  values  typical  of  videotex 
systems. 

AIL  ough  the  factors  which  contribute  to  user  response  times  may  range  in  value  from  one 
type  of  menu-based  application  to  another,  MacGregor  et  al.  (1986)  argued  that  the  search  model 
is  applicable  across  conditions.  To  date,  however,  neither  this  model  nor  the  empirical  literature 
provides  a  clear  understanding  of  how  to  derive  an  optimal  structure  given  a  particular  menu 
application.  To  increase  the  usefulness  of  the  Lee  and  MacGregor  (1985)  search  model,  the 
majority  of  the  factors  which  account  for  search/decision  and  response  times  must  be  identified, 
and  procedures  for  their  measurement  must  be  developed  and  applied  to  specific  applications 
c*  interest. 

MacGregor  et  al.  (1986)  made  progress  toward  this  end  by  examining  factors  which  influence 
user  decision  processes  for  videotex  menu  retrieval.  The  authors  hypothesized  that  the  decision 
process  used  during  menu  retrieval  Is  criterion-based.  The  criterion  is  equivalent  to  that  level 
of  probability/confidence  which  an  alternative  must  exceed  to  be  considered  as  a  choice.  Menu 
search  processes  were  seen  as  a  manifestation  of  the  same  decision  process.  A  self-terminating 
search  occurs  when  an  alternative  clearly  exceeds  the  criterion  and  is  chosen  without  further 
search.  If  an  alternative  exceeds  the  criterion  by  a  smaller  amount,  or  if  several  alternatives 
exceed  the  criterion  level,  an  exhaustive  or  redundant  search  must  be  undertaken. 

The  authors  stated  that  the  number  of  alternatives  presented  on  the  menu  page  directly 
influenced  the  criterion  level  used  during  the  decision  process.  They  argued  that  as  number  of 
alternatives  (a)  increases,  the  lower  bound  of  the  criterion  (1/a)  decreases.  From  this  criterion-based 
search  model,  several  predictions  were  generated  concerning  the  effects  that  variations  in  number 
of  alternatives  have  on  user  search  strategies  and  on  the  frequencies  of  various  types  of  errors. 

MacGregor  et  al.  (1986)  examined  the  validity  of  model  predictions  in  an  experiment  that 
incorporated  a  partial  search  procedure  in  which  subjects  examined  only  one  of  p  menu  pages 
from  a  videotex-type  menu  hierarchy  and  selected  an  alternative  on  that  page.  This  method 
provided  the  capability  of  examining  the  decision  and  search  process,  as  well  as  permitting  the 
empirical  estimation  of  parameter  values  for  the  model.  Alternatives  were  presented  either 
simultaneously  or  sequentially.  In  the  simultaneous  condition,  all  the  alternatives  were  viewed 
at  one  time.  In  the  normal  fashion.  In  the  sequential  condition,  the  initial  display  showed  only 
the  set  of  numbers  corresponding  to  menu  choices  so  that  subjects  knew  how  many  alternatives 
there  were.  Thereafter,  they  could  display  only  one  alternative  at  a  lime,  with  the  sequence 
and  timing  of  each  exposure  being  controlled  by  the  subject.  The  sequential  presentation 
procedure  permitted  the  study  of  decision  and  search  processes  employed  by  subjects.  The 
simultaneous  condition  was  used  for  comparison  purposes  to  determine  whether  subject 
performances  In  the  sequential  condition  were  Influenced  by  the  forced  sequential  search.  A  mixed 
design  was  used,  with  one  between-groups  factor  and  one  within-group  factor.  The  between 
factor  was  sequential  versus  simultaneous  presentation  ol  alternatives.  The  within  factor  was 
the  number  of  alternatives  per  page:  either  2,  4,  8,  or  16 

To  a  considerable  extent,  the  results  supported  the  predictions.  First,  It  was  predicted  that 
no  subject  would  use  any  single  strategy  but  rather,  would  employ  different  strategies.  The 
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results  indicated  (a)  that  no  subject  followed  one  strategy  to  the  exclusion  of  the  others,  and 
(b)  that  two-thirds  of  the  subjects  used  all  three  strategies.  The  model  also  predicted  that  bo*h 
self-terminating  and  redundant  searches  vvould  increase  as  the  nur'  er  of  choices  per  page 
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alternatives  were  few,  exhaustive  searches  predominated 
both  redundant  and  self-terminating  searches  tended  to 
self-terminating  searches  continued  to  increase  up  to  16 
also  increased  and  redundant  searches  dropped  off, 
suggested  that  this  may  have  been  due  to  the  effort 
alternatives.  An  alternate  explanation,  which  will  be 
report,  is  that  the  results  may  have  been  confounded 
among  the  targets  and  alternatives  across  conditions. 


.  As  the  number  of  alternatives  increased, 
increase  in  frequency.  However,  although 
alternatives  per  page,  exhaustive  searches 
contrary  to  predictions.  The  authors 
involved  in  reading  pages  with  so  many 
examined  in  greater  defail  later  in  this 
by  the  variations  in  semantic  relatedness 


AnothSi  problem  with  these  results  concerns  t.he  manner  in  which  exhaustive  and  self-terminating 
searches  were  operationally  defined  and  analyzed.  The  authors  defined  exhaustive  searches  as 
occurring  when  each  alternative  was  examined  once  and  once  only.  Self-terminating  searches 
were  defined  as  occur-ing  when  less  than  all  alternatives  were  examined  prior  to  subject  response. 
Based  on  these  definitions,  when  subjects  examined  all  alternatives  once  and  only  once  and 
then  chose  the  alternative  in  the  last  menu  position,  this  search  was  considered  exhaustive.  It 
could  be  argued  that  si  :h  cases  inflated  the  frequencies  of  exhaustive  searches  reported  in  the 
MacGregor  et  al.  (1986)  study  and  conversely  deflated  the  reported  frequencies  of  self-terminating 
searches.  As  the  number  of  alternatives  decreased,  the  percentage  of  these  cases  increased 
due  to  the  increased  probability  of  having  the  correct  alternative  occupy  the  last  position  in  the 
menu  Therefore,  it  is  reasoned  that  the  degree  of  the  inflation  of  exhaustive  searches  and 
deflation  of  self-terminating  searches  in  the  results  reported  by  MacGregor  et  al.  (1986)  was 
inversely  related  to  number  of  alternatives.  Thus,  the  reported  increase  in  self-terminating 
searches  and  decrease  in  exhaustive  searches  as  a  function  of  increasing  number  of  alternatives 
may  have  been  confounded  by  the  researchers’  operational  definitions  of  search  strategy. 

The  model  made  a  number  of  predictions  concerning  subject  decisions  and  the  incidence 
of  different  types  of  errors.  There  are  only  two  possible  correct  answers  in  a  menu  search: 
Either  the  correct  alternative  is  present  in  the  menu  and  is  chosen  (which  defines  a  "hit" 
outcome),  or  the  correct  alternative  is  not  present  in  the  menu  and  the  user  makes  a  zero 
choice,  indicating  that  none  of  the  alternatives  is  correct  (which  defines  a  "correct  rejection"). 
There  are  three  possible  incorrect  subject  responses;  The  subject  can  choose  an  incorrect 
alternative  when  the  correct  alternative  is  present  (which  defines  a  ’"commission  miss');  the 
subject  can  choose  an  incorrect  alternative  when  the  correct  alternative  is  not  present  (which 
defines  a  "false  alarm");  or  the  subject  can  make  a  zero  choice  when  the  correct  alternative  is 
present  (which  defines  an  "omission  itisss").  Table  1  summarizes  the  possible  menu  decision 
outcomes  under  this  paradigm. 


Table  1 .  Possible  Menu  Decision  Outcomes 


User  choice 

Correct  alternative 
present 

Correct  alternative 
not  present 

User  chooses  the 

Hit 

Not 

Correct  Alternative 

Possible 

Use--  chooses  an 

Commission 

False 

Incorrect  Alternative 

Miss 

Alarm 

Zero  Response 

Omission 

Correct 

(user  indicates 

Miss 

Rejection 

correct  alternative 
is  not  present  in  menu; 
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The  model  predicted  that  the  number  of  alternatives  would  inversely  affect  the  subjects’ 
frequency  of  rejecting  all  alternatives  and,  conversely,  that  the  incidence  of  selecting  some 
alternative  would  be  related  directly  to  number  of  alternatives.  Given  that  a  zero  choice  was 
made,  the  model  further  predicted  that  the  conditional  probability  of  an  omission  miss  would 
decrease  as  a  function  of  number  of  alternatives  and  the  probability  of  a  correct  rejection  would 
increase.  The  results  tended  to  support  these  predictions  (Table  2).  Significaiit  effects  due  to 
number  of  alternatives  accounted  for  about  15%  of  the  variance  in  each  analysis.  The  model 
also  predicted  that  hits  would  be  related  inversely  to  number  of  alternatives  and  that  commission 
misses  would  be  related  directly.  Both  hits  and  commission  misses  showed  some  trend  in  the 
predicted  directions,  but  the  2-  and  4-alternative  conditions  showed  a  reversal  in  trend  (Table 
3).  In  both  cases,  the  significant  effect  due  to  number  of  alternatives  accounted  for  about  46% 
of  the  variance. 


Table  2.  Zero  Choice  Outcomes  as  a  Function  of  Varying  Levels  of 
Number  of  Alternatives  (from  MacGregor  et  al.,  1986) 


Outcome 

Number  of  alternatives/page 

2  4  8  16 

Frequency  of  zero  choice 

53 

47 

27 

36 

Probability  of  omission  errors 

0.45 

0.40 

0.15 

0.14 

Probability  of  correct  rejections 

0.55 

0.60 

0.85 

0.86 

Table  3.  Alternative  Present  Choice  Outcomes  as  a  Function  of  Varying 

Levels  of  Number  of  Alternatives  (from  MacGregor  et  al.,  1986) 

Number  of  alternatives/page 

Outcome 

2 

4 

8 

16 

Frequency  of  choosing  an 

alternative 

187 

193 

213 

204 

Probability  of  commission  errors 

0.27 

0.09 

0.34 

0.39 

Probability  of  false  alarms 

0.10 

0.10 

0.12 

0.08 

Probability  of  hits 

0.63 

0.81 

0.55 

0.53 

To  a  large  extent,  the  findings  of  MacGregor  et  al.  (1986)  supported  their  contention  that 
the  search  process  can  be  viewed  usefully  as  a  criterion-based  decision  process.  The  results 
provided  evidence  that  the  number  of  alternatives  presented  on  a  menu  page  affects  the  level 

to  which  this  criterion  adjusts,  and  consequently  influences  the  search  strategy,  search  times, 

and  the  types  of  errors  that  occur.  The  authors  suggested  that  because  some  components  of 
error  were  a  direct  function  of  number  of  alternatives  and  others  were  an  indirect  function,  an 
implication  of  the  model  was  that,  overall,  errors  would  sum  to  a  quadratic  function  of  the 
number  of  alternatives,  with  intermediate  levels  leading  to  optimal  performance.  Tt.j  results 
supported  this  prediction,  with  4  to  5  alternatives  providing  the  best  arrangement. 

A  major  test  of  the  validity  of  lis  search/decision  model  rests  on  how  well  it  predicts  user 

performances  across  a  variety  of  conditions.  MacGregor  et  al.  (1986)  stressed  that  parameter 

values  which  resulted  from  their  study  were  pertinent  to  the  use  of  menus  in  large  information 
retrieval  applications  and  would  not  likely  apply  to  the  daily  user  of  a  command-type  menu. 
They  suggested  that  the  experienced  user  of  a  command  menu  is  likely  to  learn  not  only  the 
correct  alternatives  for  a  given  target  item  but  also  the  positions  of  these  alternatives  on  the 
page,  so  that  only  a  small  subset  of  alternatives  need  be  scanned  at  high  rates  of  reading  time. 
The  values  for  the  model  parameters  under  these  conditions  would  be  quite  different  from  those 
computed  from  the  MacGregor  et  al.  (1986)  study,  yielding  an  optimal  structure  with  less  depth 
and  more  breadth  than  the  one  recommended  for  videotex  menus. 
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At  issue  is  the  MacGregor  et  al.  (1986)  contention  ttiat  even  though  the  parameter  values 
may  change,  the  basic  search  model  and  its  predictions  should  hold  across  a  variety  of  menu 
applications.  The  question  arises  as  to  \whether  the  processes  involved  during  command  menu 
search  are  the  same  as  those  suggested  for  videotex  menus.  More  specifically,  if  the  optimum 
number  of  alternatives  can  be  deter.mined  by  the  same  criterion-based  processes  In  both  videotex 
and  command  menus,  then  are  variables  which  affect  the  decision  criterion  the  same  in  both 
conditions?  To  the  extent  that  this  is  true,  then  both  the  theoretical  and  empirical  literature 
suggest  factors  in  addition  to  the  number  of  menu  alternatives  that  may  play  important  roles  in 
the  menu  search  and  decision  process  for  users  at  initial  levels  of  menu  structure  familiarity. 
Those  factors  include  the  cognitive  relationships  among  targets  and  alternatives,  and  the  probabiiity 
that  the  target  is  not  subsumed  under  any  available  alternative  (i.e.,  omission  trials).  What 
follows  is  a  review  of  issues  from  the  empirical  literature  relevant  to  these  factors.  This  review 
forms  the  basis  for  recommending  that  the  search  model  be  modified  in  such  a  manner  that  it 
accounts  for  performance  effects  of  these  additional  factors. 


Overview 


Empirical  Investigations 


The  need  for  an  empirical  resolution  of  the  breadth/depth  >ssue  led  Miller  (1981)  to  investigate 
the  influence  breadth  and  depth  had  on  the  speed  and  accuracy  of  menu  selection  performances. 
Miller  (1981)  had  collcoe  students  search  for  a  word  presented  in  one  of  four  hierarchical 
structures;  (a)  menus  with  2  alternatives  at  each  of  six  levels  (2®),  (b)  menus  with  4  alternatives 
at  each  of  three  levels  (4®),  (c)  menus  with  8  alternatives  at  each  of  two  levels  (8^),  and  (d) 
menus  with  64  targets  "at  a  single  fevel  (64^).  Each  hierarchy  had  the  same  64  targets  at  its 
lowest  level  (see  Figure  1).  Targets  were  drawn  from  eight  basic  categories,  with  eight  targets 
per  category.  Each  of  the  hierarchies  was  based  on  subordinate  and  superordinate  organization 
of  the  items  contained  in  each  of  these  basic  categories.  For  the  broadest  menu  (64^),  subjects 
were  required  to  search  the  array  for  the  target  word  and  respond  by  pressing  a  button 

corresponding  to  the  location  of  a  randomly  arranged  group  of  words  containing  the  target. 

For  deeper  menus,  subjects  were  shown  names  of  categories  that  might  contain  the  target  and 
were  asked  to  select  a  sequence  of  options  which  would  lead  to  the  target  word.  Once  the 
target  was  found,  subjects  responded  by  pressing  a  button  which  identified  the  location  of  the 
target  on  the  display.  Miller  (1981)  found  that  both  speed  and  accuracy  varied  with  menu 
structure.  Subjects  were  slower  with  the  extreme  levels  of  breadth  and  depth  (i.e.,  the  64’  and 
2®  menus  -  see  Figure  2).  Error  data  also  produced  a  U-shaped  function,  with  the  least  amount 
of  error  data  produced  by  the  8^  menu  configuration.  Overall,  Miller's  (1981)  findings  would 
suggest  the  use  of  structures  of  intermediate  breadth  and  depth  in  menu  design. 

Snowberry,  Parkinson,  and  Sisson  (1983)  pointed  out  two  potential  shortcomings  of  Miller’s 
(1981)  methodology.  The  first  concerned  the  arrangement  of  items  used  In  the  broadest  menu 
structure  (i.e.,  the  64’  menu).  Though  all  other  menus  wore  constructed  with  categories  intact, 

the  64’  menu  had  targets  arranged  in  a  random  fashion.  Thus,  breadth  in  the  broadest  menu 

seemingly  was  confounded  by  the  omission  of  strict  categorical  grouping  of  display  options. 
The  second  criticism  levied  against  Miller’s  (1981)  methodology  concerned  response  requirements. 
In  all  but  the  broadest  menu,  stimuli  were  presented  along  the  right  and  left  edges  of  the 
display,  laterally  adjacent  to  response  buttons  mounted  on  the  side  edges  of  the  display  (see 
Figure  3).  Subjects  input  their  responses  by  pressing  tlie  button  adjacent  to  the  alternative 
chosen.  In  the  64’  menu  condition,  eight  pushbuttons  were  mounted  along  the  top  and  bottom 
of  the  display,  adjacent  to  tha  eight  columns  of  Items  v.'hich  contained  all  the  target  words. 
Due  to  physical  limitations  of  the  apparatus,  subjects  In  this  condition  selected  the  block  of 
eight  words  containing  the  target  word,  rather  than  Identifying  the  actual  word  Itself.  Snowberry 
ot  al.  (1983)  contended  that  because  breadth  and  response  type  were  confounded,  one  should 
not  draw  the  conclusion  that  the  slower  search  time  In  ilio  broadest  menu  condition  resulted 
from  breadth. 
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Figure  1.  Menu  Hierarchy  (adapted  from  Miller,  1981, 
and  Snowberry,  Parkinson,  and  Sisson,  1983). 
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Overall  Mean  AT  (striped  bars) 


In  a  study  designed  to  resolve  these  issues,  Snowberry  et  al.  (1983)  used  both  a  randomized 
and  a  categorized  64-item  display  to  compare  performances  on  menus  of  increasing  depth  and 
decreasing  breadth.  Response  requirements  were  held  constant  across  conditions  by  requiring 
subjects  to  respond  by  keying-in  a  two-digit  reference  number  appearing  on  the  display  directly 
adjacent  to  the  item  options  available.  Both  targets  and  hierarchical  structures  were  identical 
to  those  used  by  Miller  (1981),  as  shown  in  Figure  1.  Comparison  of  the  random  organization 
64’  menu  condition  to  other  menu  breadth/depth  conditions  produced  response  time  results 
similar  to  those  obtained  by  Miller  (1981).  Specifically,  comparison  of  response  times  across 
varying  conditions  of  breadth  and  depth  revealed  that  subjects  were  faster  with  intermediate 
breadth  and  depth  levels  and  slower  with  extreme  menu  levels.  When  category  organization 
was  held  constant  across  conditions,  however,  search  time  improved  as  depth  decreased  and 
breadth  increased  (Figure  4).  In  addition,  contrary  to  Miller’s  (1981)  findings,  accuracy  was 
found  to  improve  as  breadth  increased  (Figure  5).  This  result  held  regardless  of  the  display 
organization  (random  versus  categorized)  used  in  the  64’  condition. 
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Figure  4.  Search  Time  as  a  Function  of  Depth/Breadth 
(adapted  from  Snowberry  et  al.,  1983). 
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PERCENT  CORRECT  SEARCH 


2  4  8  64 


DISPLAY  OPTIONS 

Figure  5.  Percent  Correct  as  a  Function  of  Depth/Breadth 
(adapted  from  Snowberry  et  al.,  1983). 

The  organization  of  items  within  a  menu  was  shown  to  have  significant  consequences  on 
performances  by  the  subjects  for  the  64i  menu  condition  in  the  Snowberry  et  al.  (1983)  study. 
Other  studies  also  have  addressed  this  issue.  For  example,  Card  (1982)  found  that  for  menus 
of  18  items  there  was  an  advantage  for  an  alphabetical  menu  organization  over  categorical  and 
random  orderings.  However,  McDonald,  Stone,  and  Liebelt  (1983)  contended  that  this  finding 
may  be  applicable  only  when  users  do  not  have  a  well-formed  cognitive  organization  of  task 
categories. 

In  an  attempt  to  support  their  contention.  McDonald  et  al.  (1983)  examined  the  effects  of 
alphabetical,  categorical  and  random  organizations  on  search  performances  using  broad  (64-item) 
menus.  They  examined  five  types  of  menu  organizations  in  which  four  lists  (i.e.,  columns)  of 
16  items  each  were  combined  to  form  the  64-item  menus.  In  three  of  the  menus,  each  column 
contained  a  different  category  list  of  items.  Within  these  lists,  items  were  arranged  categorically 
(CC),  alphabetically  (CA),  or  randomly  (CR).  Within-list  categorical  orderings  (menu  CC)  were 
based  on  subjects’  similarity  ratings  for  pairings  of  items  contained  in  each  category  list.  These 
ratings  were  then  subjected  to  a  multidimensional  scaling  and  hierarchical  clustering  technique 
from  which  categorical  orderings  were  produced.  Of  the  remaining  two  menus,  one  had  all  64 
items  arranged  in  a  completely  alphabetical  order  (A)  while  the  other  had  all  items  randomly 
assigned  (R)  across  lists.  In  addition  to  the  categorization  conditions,  the  effect  of  target  type 
was  assessed  by  comparing  performance  using  explicit  targets  as  stimuli  to  performance  obtained 
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using  target  definitions  Thus,  a  5x2  factorial  design  was  usetl  which  (.tJiitamed  five  levels  of 
the  category  condition  and  two  levels  of  target  stimuli. 

Subjects  were  nired  from  a  temporary  employmefit  agency  as  general  secretarial  help.  The 
subjects'  task  in  this  study  was  to  locate  an  item  in  the  menu  and  enter  its  identification  letter 
on  the  computer  keyboard.  Results  showed  that  response  times  were  generally  faster  with 
explicit  targets  th-n  with  definitions,  nartif^nlarly  during  the  initia!  session  of  the  five-session  study 
(see  Figures  6  and  7)  With  both  target  types,  categorically  arranged  menus  (menus  CC,  CA 
and  CR)  produced  the  shortest  response  times,  wtiile  the  raridorii  organi,^aiion  of  items  across 
lists  (menu  R)  produced  the  longest  times.  Separate  analyses  weie  conducted  on  tne  response 
times  for  the  first  session  of  trials,  where  it  was  reasoned  ttiat  leorning  effects  were  at  a  minimum. 
Significant  mam  effects  were  found  for  both  item  organuatioii  and  target  type.  Within-list 
categorical  organisation  (menu  CC)  produced  trie  stiortesl  lesporise  times,  as  did  the  use  of 
explicit  targets.  Finally,  wlien  target  definitions  weie  used,  all  categurii.aily  arranged  menus 
(menus  CC,  CA  and  CR)  showed  shorter  response  times  than  those  for  the  alphabetical 
organization  (menu  A)  However,  with  explicit  targets,  alpiiaPeticai  oiganizatirin  (menu  A)  produced 
response  times  almost  as  short  as  those  produced  by  within  iiat  c  aieui  irjeal  ijrganization  (menu 
CC). 
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Figure  6.  Response  Time  as  a  Function  of  Target  Type  and 
Organization  (adapted  from  McDonald  et  al.,  1983) 


McDonald  et  al.  (1983)  interpreted  the  results  as  showing  the  superiority  of  a  categorical 
menu  organization  over  a  purely  alphabetical  one.  This  effect  was  evident  particularly  in  situations 
where  there  was  uncertainty  about  the  target.  However,  the  results  did  not  support  the  authors’ 
contention  that  it  is  possible  that  other  organizations,  particularly  alphabetical  ones,  might  be 
superior  when  users  do  not  have  a  well-formed  cognitive  organization  for  the  task  domain. 
Under  no  circumstances  did  alphabetical  organization  (menu  A)  produce  performances  superior 
to  performance  resulting  from  the  within-list  categorical  organization  (menu  CC).  Even  in  the 
first  block  of  trials  when  mental  organizations  of  the  task  domain  should  be  poorest,  performance 
in  the  CC  condition  was  superior  to  condition  A  performance.  Thus,  the  validity  of  the  McDonald 
et  al.  (1983)  contention  concerning  conditions  under  which  various  organizational  types  might 
produce  superior  performances  remains  in  question. 


Critique 

The  studies  conducted  by  Card  (1982),  Snowberry  et  al.  (1983),  and  McDonald  et  al.  (1983) 
all  support  the  contention  that  for  command-type  menu  systems,  optimal  menu  breadth  is 
somewhat  larger  than  that  which  MacGregor  et  al.  (1986)  recommended  for  videotex  menus.  In 
addition,  all  these  studies  showed  significant  effects  of  menu  categorization  on  retrieval  time  for 
broad  menu  structures.  MacGregor  et  al.  (1986)  contended  that  grouping  helps  for  precisely 
the  same  reason  that  hierarchical  organization  helps:  It  restricts  the  options  that  have  to  be 
searched.  In  either  case,  however,  search  is  restricted  only  to  the  extent  that  the  structure 
reflects  the  user's  perceived  relationships  among  targets  and  menu  alternatives.  For  frequent 
users  of  a  system,  learning  the  menu  structure  can  be  aided  by  designing  the  system  to  reflect 
preconceived  menu  item  relationships.  For  infrequent  system  users,  designing  the  system  to 
reflect  user  perceptions  of  these  reiationships  becomes  even  more  critical. 

The  effect  that  user-perceived  relationships  among  targets  and  menu  alternatives  has  on 
performance  has  received  little  empirical  attention.  In  simple  command  menus  of  the  type  used 
by  Miller  (1981),  Card  (1982),  Snowberry  et  al.  (1983),  and  McDonald  et  al.  (1983),  cognitive 
relationships  might  best  be  thought  of  in  terms  of  semantic  similarity  or  distances  among  menu 
items.  The  lack  of  attention  to  and.  perhaps  more  importantly,  failure  to  control  this  factor  may 
represent  a  possible  confounding  of  the  findings  in  several  menu  optimization  studies.  In  support 
of  this  argument  are  semantic  memory  models  (e.g.,  Collins  &  Quillian,  1969;  Rips,  Shoben,  & 
Smith,  1973)  which  contend  that  the  semantic  distance  between  a  subset  category  and  its 
member  is  closer  than  the  distance  between  the  superset  category  and  the  member  (e.g.,  robin 
and  bird  versus  robin  and  animal).  Thus,  the  results  of  breadth/depth  studies  in  which  semantic 
distances  between  targets  and  category  sub/supersets  were  not  controlled  may  have  been 
confounded  by  this  factor.  For  example,  recall  that  both  Miller  (1981)  and  Snowberry  et  al. 
(1983)  varied  breadth  and  depth  by  manipulating  subordinate  and  superordinate  organization  of 
64  targets  drawn  from  eight  basic  categories  (review  Figure  1).  As  menu  depth  varied,  so  did 
the  semantic  distances  from  the  target  to  the  menu  alternatives.  To  the  extent  that  semantic 
distance  variations  may  have  affected  performances,  performance  variations  that  were  attributed 
to  breadth/depth  manipulations  may  have  been  confounded. 

Another  factor  not  formally  considered  in  the  search  model  is  the  effect  of  variations  in 
omission  probability;  that  is,  the  probability  that  the  target  is  not  subsumed  under  any  of  the 
available  alternatives.  For  discriminate  decision  processes,  it  is  known  from  signal  detection 
theory  that  as  probability  of  an  omission  trial  varies,  decision  criterion  levels  shift  (Green  & 
Swets,  1966).  If,  as  suggested  by  MacGregor  et  al.  (1986),  the  search  model  involves  a 
criterion-based  decision  process,  then  omission  probability  should  have  a  significant  effect  on 
user  criterion  levels. 

The  above  discussions  regarding  semantic  relationships  among  menu  items  and  omission 
probability  lead  to  the  following  hypotheses  concerning  processes  involved  in  the  search  model: 
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(a)  For  simple  command-type  menu  structures,  semantic  relationships  among  targets  and 
alternatives  can  significantly  influence  the  search  and  decision  processes  involved  in  menu  task 
performance;  and  (b)  similar  to  tfie  manner  in  which  variations  in  the  number  of  menu  alternatives 
affects  ttie  decision  cnteiion,  omission  probability  also  significantly  affects  the  decision  criterion 
used  in  the  decision  process  The  concept  of  omission  probability  is  relatively  straightforward, 
but  conceptualizing  and  measuring  semantic  relationships  among  targets  and  alternatives  are 
more  complex  Theoretical  models  and  empirical  findings,  in  the  areas  of  categorization  and 
the  organization  and  sniicture  of  semantic  memory,  provide  clues  as  to  how  to  best  address 
this  problem  The  next  s-'Hdicjn  firovides  a  general  review  of  this  material,  along  with  implications 
concerning  ttm  riefinition  and  measurement  of  menu  ifem  relationships 


Semantic  Information  Processing 

The  separation  of  long  term  memory  into  episodic  and  semantic  memory  components  was 
first  suggested  by  Tulving  (197?).  f-le  conceptualized  two  separate  but  interrelated  systems  for 
the  storage  of  differing  types  of  information:  The  episodic  component  stored  temporal,  sequential 
information  concerning  instances  and  events,  as  well  as  biographical  information;  the  semantic 
component  was  defined  as  memory  for  language  and  language  syntax  A  general  evaluation 
of  research  and  theoretical  positions  concerning  the  nature  of  the  semantic  memory  component 
is  given  below. 


General  Evaluation 

The  primary  findings  from  studies  which  have  focused  on  the  processes  involved  in 
categorization  and  hio  mganization  ot  semantic  memory  can  be  summarized  as  follows: 

1  Effects  of  graded  structure  of  category  members.  A  central  issue  in  categorization 
research  has  been  ffie  finding  that  categories  possess  graded  structure.  Graded  structure  refers 
to  the  degree  to  which  members  provide  a  good  example  of  their  various  category  classes. 
For  a  given  level  of  concepts  within  a  semantic  hierarchy,  graded  structure  has  been  shown  to 
be  strongly  related  to  both  the  speed  and  accuracy  ot  verifying  class  membership  (Rips  et  al. 
1973:  Rosch,  1973,  1975,  Shoben.  1976;  Smith,  1978;  Smith,  Balzano,  &  Walker.  1978) 

2  Effects  of  graded  st'urtnre  of  category  nonmembers  Semantic  relationships  between 
categories  and  nonmernbers  are  yraried  in  much  the  same  manner  as  are  members  of  a  category. 
Graded  structure  has  been  shown  to  affect  both  speed  and  accuracy  in  verifying  nonmembers 
of  a  class  (Barsaloii,  1983,  Sholien,  1976,  Smith  et  al,,  1978). 

3  Category  size  effects  Both  speed  and  accuracy  in  semantic  verification  tasks  have  been 
shown  to  he  significantly  related  to  the  level  at  which  the  category  resides  within  a  network 
hierarchy  (Collins  H  Quillian.  1969,  1970)  The  reversal  of  this  effect  can  occur  for  category 
structures  in  which  subordinate  level  concepts  have  lower  semantic  similarity  to  exemplars  than 
do  SLiperordinafe  Ip.'eis  (Rips  et  al..  1973,  Smith  et  al  ,  1978). 

A  typical  menu  navigation  task  can  he  conceptualized  in  terms  of  the  multiple  verification 
of  category  relatedness  of  individual  alternatives  presented  on  a  menu  page  to  a  particular 
target.  Although  no  studies  could  be  found  in  the  literature  which  bear  directly  on  the  problem 
of  defining  those  semantic  variables  which  affect  performance  on  such  a  task,  the  above  findings 
from  semantic  venfirafion  studies  can  be  generalized  to  the  multiple  verification  problem  in  an 
attempt  to  provide  a  fran’^vyork  for  considering  decision  processes  that  may  be  involved  in 
menu  task  (lerformanr  e  tfie  varying  semantic  relationships  between  the  target  and  individual 
alternatives  on  ,r  merni  (  .••gf'  •  .nn  Im  exfiressed  in  terms  of  the  graded  structure  of  category 
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members  and  nonmembers,  as  well  as  each  alternative’s  level  within  the  menu  hierarchy.  An 
implication  from  the  above  findings  is  that  menu  item  selection  will  at  least  partially  depend  on 
the  semantic  similarity  between  the  target  and  the  set  of  alternatives  presented  on  the  menu 
page. 

To  the  extent  to  which  the  relationships  among  targets  and  menu  set  alternatives  are  graded 
both  within  and  across  levels  of  the  menu  hierarchy,  the  problem  of  evaluating  their  effects  on 
menu  task  performance  is  reduced  to  one  of  measuring  these  varying  relationships  by  empirical 
study.  The  next  section  provides  an  overview  of  techniques  that  have  been  employed  as 
measures  of  graded  structure  and  semantic  similarity  in  categorization  and  semantic  memory 
research. 


Measurement  of  Categorical  Relationships 

The  requirements  of  the  present  investigation  called  for  measures  which  reflect  the  graded 
relationships  among  targets  and  menu  alternatives.  Research  in  this  area  has  shown  that,  to 
a  large  extent,  the  graded  structure  of  an  exemplar  can  be  determined  in  terms  of  its  degree 
of  similarity  to  other  category  members  and  its  degree  of  dissimilarity  to  category  nonmembers. 
These  findings  suggest  that  measurement  requirements  for  the  present  work  can  be  satisfied  by 
deriving  valid  measures  of  semantic  similarity  for  targets  with  respect  to  a  particular  set  of 
alternatives  presented  on  a  menu  page  for  selection. 

Measures  of  semantic  similarity  are  usually  obtained  by  having  subjects  make  pairwise 
comparisons  of  all  items  in  a  stimulus  set  or  of  a  subset  of  these  item  combinations  and  provide 
similarity  ratings  based  on  subjective  evaluation.  Ratings  are  then  averaged  across  subjects 
and  used  as  predictors  of  response  times  and  error  rates  on  experimental  tasks.  Rips  et  al. 
(1973)  introduced  this  procedure  for  use  in  predicting  semantic  verification  response  times  and 
it  has  subsequently  been  used  in  several  studies  of  semantic  memory  (e.g.,  Barsalou,  1985; 
Rosch,  1975;  Shoben,  1976). 

In  addition  to  demonstrating  the  significant  predictive  capability  of  raw  rating  averages,  both 
Rips  et  al.  (1973)  and  Shoben  (1976)  demonstrated  the  predictive  potential  of  semantic  distances 
derived  from  the  multidimensional  scaling  (MDS)  of  subjects’  similarity  ratings.  MDS  takes  as 
input  the  semantic  similarity  ratings  of  all  possible  pairings  of  the  stimulus  set,  and  yields  as 
output  an  Euclidean  solution  in  n  dimensions  where  the  orientation  of  these  dimensions  is  fixed 
by  the  scaling  procedure  itself.  The  advantages  of  the  use  of  the  MDS-derived  distance  values 
over  raw  rating  averages  are  both  substantive  as  well  as  methodological. 

The  substantive  contribution  of  MDS  solutions  is  that  they  offer  a  method  for  recovering 
underlying  structural  features  of  scaled  items.  This  may  be  achieved  through  the  identification 
and  interpretation  of  directions  or  axes  through  the  spatial  representation.  Thus,  moving  farther 
along  some  particular  direction,  points  are  successively  encountered  which  correspond  to  objects 
that  possess  more  and  more  of  some  particular,  identifiable  property.  Such  results  have  been 
used  to  describe  the  characteristic  features  of  category  terms  in  semantic  memory  studies  (e  g.. 
Rips  et  al.,  1973;  Shoben,  1976). 

The  methodological  contribution  of  MDS  is  twofold.  First,  MDS  provides  a  less  noisy  measure 
of  relatedness  than  do  raw  rating  averages.  Noise  here  refers  to  error  that  distorts  the 
measurement  of  the  relationships  among  items  in  the  stimulus  set.  MDS  proponents  claim  that 
MDS  provides  a  more  meaningful  and  interpretive  configuration  by  smoothing  out  much  of  this 
noise  through  its  computations.  For  example,  both  Rips  et  al.  (1973)  and  Shoben  (1976) 
demonstrated  that  in  certain  semantic  memory  tasks,  the  Euclidean  distances  derived  from  MDS 
surpassed  the  raw  rating  averages  in  predicting  subject  performances.  The  authors  argued  that 
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the  MDS  solutions  reduced  the  noise  in  the  data  by  forcing  the  configuration  into  only  the  few 
critical  dimensions  which  were  reflective  of  feature  similarity  among  stimuli. 

The  other  methodological  advantage  that  certain  MDS  techniques  hold  over  raw  rating 
averages  concerns  the  metric  properties  associated  with  MDS.  MDS  can  be  thought  of  as  a 
means  of  recovering  metric  configurations  from  nonmetric  rating  information.  The  techniques 
used  in  MDS  assume  that  within  a  matrix  of  rating  sums  there  exists  a  true  underlying  configuration 
of  points  in  n-dimensional  Euclidean  space  that  can  be  ascertained  by  the  linear  ordering  of 
interpoint  distances  (Kruskal,  1964).  MDS  procedures  are  designed  to  recover  this  underlying 
configuration.  Though  the  nonmetric  properties  of  raw  ratings  tend  to  make  questionable  the 
use  of  certain  types  of  arithmetic  operations,  the  metric  properties  associated  with  the  MDS 
configuration  enable  the  use  of  parametric  operations  without  fear  of  assumption  violation. 

For  example,  later  in  this  report,  a  technique  will  be  detailed  which  attempts  to  combine 
two  separate  sets  of  pairwise  similarity  distance  values  based  on  distance  values  between  items 
common  to  each  set.  The  technique  produces  a  single  configuration  containing  all  items  from 
both  sets  through  the  scaling,  translation  and  reorientation  of  items  in  the  second  set  into 
maximal  congruence  with  common  items  in  the  first  set.  Although  the  operations  involved  in 
this  transformation  might  be  questionable  for  use  with  nonmetric  data,  they  are  perfectly  justifiable 
for  use  with  MDS  solutions. 

The  substantive  advantages  of  MDS  are  not  particularly  relevant  to  the  present  work.  The 
main  interest  here  is  in  obtaining  valid  measures  of  the  semantic  relationships  among  sets  of 
items.  Thus,  the  identification  of  underlying  dimensional  characteristics  is  not  of  real  consequence. 
The  methodological  advantages  of  MDS,  however,  are  of  significant  interest.  Particularly,  the 
metric  properties  associated  with  MDS  solutions  provide  a  substantial  advantage  over  raw  rating 
averages.  Therefore,  the  decision  was  made  to  employ  distance  values  derived  from  MDS 
analysis  of  rating  averages  as  measures  of  the  semantic  relationships  among  menu  set  stimuli. 
The  specific  procedures  and  results  of  this  work  are  detailed  in  Section  III  of  this  report. 


Synopsis 

The  material  reviewed  in  this  section  suggests  that  the  relationships  among  targets  and  menu 
alternatives  in  a  computer  menu  selection  task,  such  as  the  one  used  in  the  study  by  MacGregor 
et  al.  (1986),  can  be  conceptualized  in  terms  of  semantic  distances  among  menu  set  items 
derived  from  MDS  solutions  of  similarity  ratings.  In  the  present  investigation,  these  relationships 
were  defined  and  measured  operationally  as  two  distinct  measures.  The  first  measure  reflected 
the  relationship  between  the  target  and  the  correct  alternative  in  the  menu  display  ("semantic 
distance  to  correct"  or  Dc).  This  measure  was  operationally  defined  as  the  semantic  distance 
between  these  items  derived  from  MDS  solutions  of  similarity  ratings.  The  second  measure 
reflected  the  amount  of  competition  for  selection  produced  by  the  incorrect  alternatives  presented 
in  the  menu  display  ("semantic  distance  to  incorrect"  or  Di).  The  Di  measure  was  defined 
operationally  in  more  than  one  manner.  The  competition  produced  by  incorrect  alternatives  was 
defined  as  either  a  distance  measure  from  correct  to  incorrect  alternatives  or  a  distance  measure 
from  the  target  to  incorrect  alternatives.  Note  that  as  either  of  these  distance  measures  increases, 
the  selection  competition  produced  by  incorrect  alternatives  in  the  menu  decreases.  In  addition, 
each  measure  was  operationally  defined  either  as  a  measure  of  the  central  tendency  among 
distance  values  for  all  competing  alternatives  or  as  a  single  measure  of  distance  to  the  semantically 
nearest  neighbor  in  the  menu  display. 

It  was  decided  to  attempt  to  derive  distance  measures  among  menu  set  items  that  would 
allow  for  the  computation  of  all  these  measurement  variants.  Resolution  as  to  which  measure 
best  accounts  for  performance  variations  could  then  be  determined  empirically.  Section  III  details 
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the  results  of  this  work.  For  the  moment.  Di  is  defined  as  a  measuie  of  semantic  distance 
that  is  inversely  proportional  to  the  level  of  selection  competition  produced  by  incorrect  alternatives. 
Thus,  as  Di  decreases,  competition  for  selection  increases. 

From  the  review  of  semantic  memory  research,  changes  in  Dc  and  Di  are  predicted  to  affect 
both  menu  selection  search  time  and  accuracy  Specifically,  the  lower  the  value  of  Dc  (reflecting 
a  high  degree  of  similarity  between  target  and  correct  alternative),  the  lower  will  be  both  the 
processing  time  and  the  probability  of  error  resulting  from  the  evaluation  of  a  correct  alternative 
Conversely,  the  lower  the  value  of  Di  (reflecting  a  high  degfee  oi  seiection  competition  from 
incorrect  alternatives),  the  greater  will  be  the  processing  time  and  the  probability  of  error  resulting 
from  the  evaluation  of  individual  incoirect  alteriiatives 

The  effects  that  semantic  similarities  among  targets  and  alternatives  have  on  selection  speed 
and  accuracy  can  be  conceptualized  in  terms  of  variations  in  tfie  distributions  of  relat'^dness 
among  targets  and  alternatives  with  respect  to  selection  critena  Consider,  tor  example,  the 
two-stage  semantic  memory  model  proposed  by  Smith  Sholron  and  Rips  (1974).  Figure  8 
provides  a  schematic  representation  of  the  hypothesized  e'^ects  that  variations  in  relatedness 
distributions  have  on  response  processes  during  a  category  verification  task.  The  figure  illustrates 
the  process  by  which  a  subject  determines  whether  test  instances  are  members  of  a  particular 
category  The  two  distributions  on  the  rigtit  side  represent  probability  distributions  for  instances 
of  typical  and  atypical  category  members  The  two  other  distributions  represent  distributions 
'or  related  and  unrelated  nonmembers.  According  to  Smith  et  a'  's  (1974)  model,  the  subject 
performs  this  verification  task  by  first  comparing  the  instance  to  the  category  and  ttien  by 
determining  the  overall  semantic  similarity,  x,  of  tfie  category  to  the  instance  Cq  and  C,  are 
selection  criteria  used  to  determine  whetfier  to  execute  a  fast  true  ovponse  (x  >  C,)  or  a  fast 
false  response  (x  •-  Cq).  or  to  go  on  to  a  second  comparison  stage  (Cq  <  x  <  C,).  Smith 
et  al  (1974)  tested  predicted  penormances  of  this  model  and  showed  that,  to  a  la'^ge  extent, 
results  supported  the  conceptualization  of  semantic  similarity  effects  in  the  manner  specified. 


Semantic  Relatedness  -  > 

Figure  8  Hypothetical  Distribution  of  Overall  Similarity  Values  Between  Targets 
and  Category  Names  (adapted  from  Smith  et  al.,  1974) 

The  effects  of  semantic  similarities  amotig  menu  set  items  in  a  menu  selection  task  can  be 
conceptualized  in  a  similar  manner  by  generalizing  from  the  sii  gle  instance/category  verification 
problem  In  the  menu  selection  task  where  the  subject  a'ternpts  to  determine  under  which  of 
the  menu  alternatives  the  target  is  subsumed,  the  axis  in  Figure  8  would  no  longer  represent 
the  relatedness  of  the  target  to  a  single  category  Rather,  the  axis  v/ould  represent  a  more 
general  function  of  the  degree  of  semantic  relatedness  that  can  be  used  to  illustrate  the 
relationship  between  the  target  and  any  alternative  cor',.,idered  for  selci  lion  In  addition,  rather 
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than  representing  probabilities  for  members  and  nonmembers  of  a  single  category,  the  distributions 
would  represent  probabilities  for  members  of  the  correct  alternative  (illustrated  by  the  two 
distributions  to  the  right)  and  the  overall  probability  for  nonmembers  of  all  incorrect  alternatives 
(illustrates  by  the  distributions  to  the  left).  Assuming  that  Dc  and  Di  values  have  been  scaled 
to  represent  corrmon  metrics  of  semantic  relatedness,  the  axis  in  Figure  8  can  be  thought  of 
as  representing  these  same  metrics.  (Note  that  both  Dc  and  Di  decrease  as  semantic  simiiarity 
increases,  or  as  values  move  left  to  right  along  the  axis.) 

The  probability  distribution  representing  the  semantic  similarity  between  correct  alternatives 
and  typical  targets  is  illustrated  in  Figure  8  by  the  curve  farthest  to  the  right.  The  mean  Dc 
value  associated  with  this  distribution  would  be  relatively  low,  reflecting  a  high  degree  of  semantic 
reiatedness.  The  probability  distribution  representing  the  relationship  between  correct  alternatives 
and  atypical  targets  is  illustrated  by  the  second  curve  from  the  right.  The  mean  Dc  value  for 
this  distribution  would  be  somewhat  higher  than  that  for  the  distribution  labeled  "Typical,"  reflecting 
decreased  semantic  relatedness.  The  probability  distributions  of  semantic  relatedness  of  targets 
to  related  and  unrelated  incorrect  alternatives  are  represented  by  the  two  curves  on  the  left. 
The  mean  Di  values  for  these  distributions  would  be  relatively  higher  than  the  mean  Dc  values 
associated  with  the  other  two  distributions,  reflecting  decreased  reiatedness.  Ignoring  for  the 
moment  the  placement  of  selection  criteria,  it  is  apparent  from  this  illustration  that  variations  in 
the  probability  distributions  of  Dc  and  Di  values  essentially  amount  to  variations  in  the  discriminability 
among  correct  and  incorrect  alternatives. 

In  terms  of  Smith  et  al.'s  (1974)  model  of  semantic  memory,  such  variations  have  been 
shown  to  affect  processing  times  of  individual  alternatives,  as  well  as  response  accuracy,  in 
category  verification  tasks.  In  terms  of  the  menu  search  model  proposed  by  MacGregor  et  al. 
(1986).  variations  in  Dc  and  Di  distributions  might  be  expected  to  affect  menu  search  strategies 
as  well.  T^  the  extent  that  such  search  strategy  variations  exist,  as  was  evidenced  by  the 
results  of  MacGregor  et  al.’s  986)  experiment,  a  model  analogous  to  the  one  proposed  by 
Smith  et  al.  (1974)  is  proposed  that  predicts  such  search  strategy  variations  as  being  partially 
due  to  variations  in  the  discriminability  among  menu  alternatives.  Details  of  this  model  are 
presented  in  Section  V  and  are  used  in  Section  VI  to  explicate  the  results  of  the  menu  task 
experiment  described  in  the  next  section. 


Research  Rationale 

The  background  literature  reviewed  in  this  section  suggests  that  several  factors  potentially 
contribute  to  the  creation  of  an  optimal  menu  design;  (a)  the  number  of  alternatives  displayed 
on  the  menu,  (b)  the  semantic  relationships  among  targets  and  menu  item  alternatives,  and  (c) 
the  probab'Mty  of  a  correct  alternative  omission.  Though  the  lit  rature  has  shown  these  factors 
to  be  predictive  of  performance  on  menu  or  menu-related  tasks,  the  effects  these  factors  have 
on  menu  task  performance  have  never  been  studied  under  conditions  where  all  factors  are 
manipulated  within  a  single  experimental  design.  Such  an  experiment  was  the  focus  of  the 
present  investigation.  The  design  permitted  the  assessment  of  effects  each  factor  had  on  task 
performance.  In  addition,  through  the  independent  estimation  of  menu  item  semantic  .elationships, 
potential  confounding  due  to  experimental  manipulation  of  omission  probability  and  number  of 
alternatives  was  also  evaluated. 

The  first  step  was  to  select  a  menu  set  appropriate  for  the  current  investigation  and  to 
quantify  semantic  relationships  among  menu  item  members.  Menu  set  selection  was  based  on 
requirements  that  the  menu  represent  a  command-type  menu  hierarchy  with  a  categorical  structure 
that  could  be  manipulated  logically  for  experimental  purposes.  An  additional  requirement  was 
that  semantic  relationships  among  the  menu  items  within  the  set  had  to  be  quantified.  This 
requirement  stemmed  from  the  need  to  verify  that  experimental  manipulations  made  to  menu 
stimuli  actually  resulted  in  variations  of  semantic  relationships  among  menu  targets  and  alternatives. 
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Miller’s  (1981)  menu  hierarchy  (Figure  1)  fulfilled  the  requirements  for  a  command-type  menu, 
with  appropriate  hierarchical  relationships  among  its  members.  In  addition,  an  extensive 
performance  database  has  been  developed  for  this  menu  through  various  experiments  reported 
in  the  literature  and  could  be  used  to  help  validate  predictions  and  results  of  the  present 
investigation.  Although  the  concept  and  measurement  of  the  number  of  menu  alternatives  and 
omission  probability  are  relatively  straightforward,  measurement  of  semantic  relationships  among 
targets  and  menu  alternatives  is  more  complex.  The  requirement  to  quantify  the  relationships 
among  the  items  contained  in  the  menu  set  was  satisfied  through  the  collection  of  similarity 
ratings  from  groups  of  subjects  for  pairwise  combinations  of  menu  stimuli.  Distance  values 
were  estimated  from  the  rating  data  using  MDS  procedures.  Upon  completion  of  the  scaling 
work,  the  menu  set  then  was  used  in  an  experiment  designed  to  quantify  the  performance 
effects  of  the  four  factors  of  interest  (Dc,  Di,  omission  probability  and  number  of  alternatives). 
Details  of  the  scaling  and  experimental  methods  are  described  in  the  next  section.  The  results 
of  this  v/ork  are  explicated  by  means  of  a  two-criterion  menu  model  which  is  detailed  in  Section  V 
of  this  report. 


III.  METHODS 
.  Purpose 


Tfio  purpose  of  the  present  experiment  was  to  quantify  ilie  performance  effects  of  four  menu 
design  factors  on  subject  performances.  The  four  factors  were:  (a)  the  hierarchical  level  of 
the  menu  page  (i.e.,  the  level  within  the  menu  hierarchy  from  which  the  menu  was  derived): 
(b)  the  hierarchical  relationship  or  "nesting"  that  existed  among  menu  set  alternatives  (i.e.,  the 
level  in  the  hierarchy  at  which  all  menu  alternatives  had  a  common  superordinate):  (c)  the 
number  of  alternatives  that  were  presented  on  the  menu:  aiid  (d)  the  probability  that  the  correct 
alternative  was  omitted  from  the  menu.  Hereafter,  those  factors  will  be  referred  to  as  the 
Hierarchy,  Nesting,  Number  of  Alternatives,  and  Omission  Probability  factors,  respectively.  A 
final  factor  used  in  this  experiment  was  sequential  versus  simultaneous  presentation  of  menu 
stimuli,  Hereafter,  this  factor  will  be  referred  to  as  the  Presentation  Mode  factor.  The  sequential 
condition  provided  direct  measures  of  search  strategies.  Most  command  menu  implementations, 
however,  involve  simultaneous  presentation  of  alternatives.  Use  of  both  conditions  allowed 
comparison  of  response  accuracy  data  obtained  for  each  condition. 


Determining  Semantic  Relationships  Among  Menu  Stimuli 

The  task  stimuli  used  in  this  investigation  represented  i  command-type  menu  hierarchy  with 
a  categorical  structure  that  could  bo  manipulated  logically  for  experimental  purposes.  Additionally, 
semantic  relationships  among  the  stimulus  items  within  the  set  had  to  be  quantified.  This  second 
requirement  stemmed  from  the  need  to  verify  that  experimental  manipulations  made  to  menu 
stimuli  actually  resulted  in  variations  of  semantic  relationships  among  menu  targets  and  alternatives. 

Miller’s  (1981)  menu  hierarchy  (Figure  1)  met  the  recjuirements  for  a  command-type  menu 
with  appropriate  hierarchical  relationships.  In  addition,  an  extensive  performance  data  base  had 
been  developed  using  this  menu  that  could  be  used  lor  i  omparison  to  predictions  and  results 
of  the  present  Investigation.  The  requirement  to  quantify  the  relationships  among  the  items 
contained  in  this  set  was  satisfied  through  a  scaling  procedure  in  which  pairwise  ratings  of 
semantic  similarity  for  menu  sot  Items  were  collected  from  groups  of  subjects  and  then  scaled 
using  multidimensional  scaling  (MDS)  techniques.  A  summary  of  the  general  approach  and 
results  of  these  scaling  efforts  is  described  below.’ 


’  A  detailed  sumtTiary  ol  the  semantic  scallng.ettort  appears  In  Pierce  (1989!. 
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Menu  Set  Selection 


The  design  for  the  menu  task  experiment  required  the  selection  and  scaling  of  only  a  subset 
of  the  items  in  Miller’s  (1981)  original  hierarchy  (see  Figure  1).  The  newly  formed  menu  set 
contained  all  64  targets  from  Level  VI,  the  32  items  from  Level  V,  and  the  eight  items  from 
Level  III.  Items  contained  in  the  resulting  menu  set  are  shown  in  Figure  9. 

All  items  from  Level  VI  of  Miller’s  (1981)  hierarchy  were  used  as  menu  targets.  Targets  are 
those  category  items  labeled  1  through  8  in  Figure  9.  The  eight  items  taken  from  Level  III  of 
Miller’s  (1981)  hierarchy  were  used  to  construct  menus  with  alternatives  'naving  a  distant  hierarchical 
relationship  to  Level  VI  targets.  Level  III  alternatives  are  listed  as  item  13  of  each  category  set 
in  Figure  9.  The  32  items  taken  from  Level  V  of  the  hierarchy  were  used  to  construct  menus 
with  alternatives  having  a  close  hierarchical  relationship  to  Level  VI  targets.  Level  V  alternatives 
are  numbered  9  through  12  in  each  category  listed  in  Figure  9. 


Scaling  Requirements 

An  assessment  of  the  impact  that  menu  item  relationships  had  on  task  performances  required 
a  measure  of  the  semantic  relatedness  between  the  targets  and  the  correct  menu  alternatives 
(measures  of  Dc),  and  measures  reflecting  the  amount  of  selection  competition  produced  by 
incorrect  menu  alternatives  (measures  of  Di).  A  series  of  data  collection  procedures  were 
designed  such  that  these  scaling  requirements  could  be  met  in  one  of  two  ways. 

The  first  approach  was  to  develop  measures  of  the  distances  among  all  items  that  formed 
the  menu  set.  From  these  measures,  distances  from  targets  to  all  alternatives  would  be  available 
for  computing  target-to-correct-alternative  similarity  measures,  as  well  as  a  variety  of  measures 
reflecting  the  semantic  relatedness  between  targets  and  incorrect  menu  alternatives.  Traditional 
approaches  to  multidimensional  scaling  (MDS)  of  semantic  similarities  among  members  of  a  word 
set  require  ratings  for  all  possible  pairwise  combinations  of  every  member  of  the  set.  The  rating 
data  then  are  scaled  to  a  best-fit  multidimensional  configuration  using  one  of  several  statistical 
programs  designed  for  this  purpose  (see  Schiffman,  Reynolds,  &  Young,  1981). 

To  obtain  rating  data  for  all  possible  pairwise  combinations  of  the  104  items  in  the  current 
menu  set  would  require  5,356  comparisons  per  subject.  An  alternative  scaling  approach  was 
examined  which  produced  a  set  of  distance  values  for  all  possible  pairwise  combinations  of 
menu  items  from  a  much  smaller  subset  of  pairwise  similarity  ratings  (Homa  &  Konrad,  1987; 
Konrad,  1988).  The  technique  was  used  to  "conjoin"  MDS  solutions  of  pairwise  similarity  data 
collected  for  subsets  of  the  104-item  menu  set.  Each  subset  had  several  items  in  common 
with  other  subsets  to  which  it  was  conjoined.  The  conjoining  algorithm  took  the  two  subsets 
of  scaled  elements,  defined  by  n-dimensional  coordinates,  and  used  the  common  item  coordinates 
of  one  subset  to  scale,  translate,  and  orient  those  coordinates  into  maximal  congruence  with 
the  coordinates  of  the  other  subset. 

There  were  three  main  operations  involved  in  producing  the  conjoined  soiution.  First,  the 
algorithm  equated  the  extent  or  volume  of  the  common  elements,  resulting  in  an 
expansion/contraction  of  one  of  the  two  scaled  spaces.  Second,  the  common  elements  were 
translated  such  that,  in  the  final  solution,  they  were  centered  around  the  same  point  in  space. 
Finally,  the  two  spaces  were  conjoined  such  that  the  pairwise  Euclidean  distances  of  the  common 
elements  were  minimized.  Although  this  technique  greatly  reduced  the  number  of  pairwise 
comparisons  required  of  individual  subjects  to  produce  its  soiution,  the  technique  was  new  and 
the  validity  of  its  results  had  yet  to  be  tested  fully.  For  this  reason,  data  collection  procedures 
were  designed  to  provide  not  only  measures  of  inter-item  distances  among  menu  set  members, 
but  also  a  means  for  testing  the  validity  of  the  resulting  solutions. 
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Gravity  1.  Spine  1.  Salmon  1,  Rose 

Magnetism  2.  Limbs  2.  Trout  2.  Daisy 


21 


Figure  9.  Menu  Set  Items  Derived  from  Levels  III,  V,  and  VI  of  Miller’s  (1981)  Hierarchy. 


Scaling  procedures  also  were  designed  such  that,  should  the  validity  of  the  conjoining 
procedures  prove  questionable,  a  second  approach  to  meeting  scaling  requirements  could  be 
implemented  without  the  need  for  additional  rating  data.  This  approach  used  the  MDS  soiutions 
for  the  menu  item  subsets  without  empioying  the  conjoinii  g  procedure.  The  approach  produced 
estimates  of  semantic  distances  among  items  within  category  sets,  as  weli  as  estimates  of  the 
semantic  distances  among  the  40  Levels  III  and  V  alternatives  used  to  construct  menu  pages 
for  the  menu  task  experiment.  Thus,  distances  from  targets  to  correct  alternatives  within  menus 
were  estimated  under  this  approach,  as  were  distafices  from  the  correct  menu  page  alternative 
to  incorrect  ones.  The  main  drawback  of  this  approach  was  that  it  did  not  produce  distance 
estimates  from  the  target  lo  alternatives  outside  the  targ<'t's  immediate  category  set.  Because, 
as  will  be  detailed  later  in  this  section,  some  of  the  menu  pages  were  constructed  using 
alternatives  from  outside  the  target’s  own  category,  this  seating  approach  did  not  allow  the 
computation  of  distances  from  targets  to  incorrect  alternatives  contained  in  these  menu  pages. 
Measures  of  selection  competition  produced  by  incorrect  alternatives  that  could  be  computed 
from  this  second  approach,  tfierefore,  were  limited  to  include  only  estimates  of  the  distances 
from  the  correct  menu  alternative  to  incorrect  ones. 


Results  Summary 

Measures  of  congruence  between  those  solutions  wfiich  used  the  conjoin  procedure  and 
those  solutions  which  did  not  Indicated  that  the  conjoin  solutions  were  of  questionable  validity. 
Therefore,  it  was  decided  to  use  distance  measures  derived  directly  from  MDS  solutions  of 
subject  rating  averages  for  separate  sets  of  pairwise  combinations  of  menu  items.  Similarity 
ratings  were  obtained  for  all  possible  combinations  of  the  40  alternatives  from  Levels  III  and  V 
(Figure  9).  In  addition,  separate  rating  sets  were  obtained  for  each  grouping  of  category  set 
items.  Thus,  the  13-item  category  sets  also  depicted  in  Figure  9  formed  eight  groups  of  pairwise 
combinations  wfiich  were  individually  rated  for  similarity  l)y  subject  groups. 

Individual  MDS  solutions  were  derived  for  these  nine  rating  sets  (the  set  of  40  alternatives 
from  Levels  III  and  V  and  the  eight  13-item  category  sets).  Resulting  solutions  for  each  of  the 
eight  13-item  category  sets  then  were  scaled  to  a  set  of  interpoint  distances  (IPDs)  derived 
from  items  common  lo  the  set  of  40  alternatives  from  Levels  III  and  V.  The  resulting  IPD  values 
for  the  MDS  solution  of  the  set  of  40  alternative.*)  provided  semantic  simiiarity  estimates  among 
correct  and  incorrect  alternatives  used  to  construct  menu  set  displays  for  the  menu  task 
experiment.  These  measures  were  used  to  compute  values  of  the  semantic  distance  between 
correct  and  incorrect  menu  alternatives  (tfie  Di  distance  measure).  iPD  values  derived  from  the 
scaled  MDS  solutions  for  the  eight  13-item  category  sots  were  used  to  compute  semantic 
similarity  estimates  among  targets  and  correct  alternatives  (the  Dc  distance  values).  The  resulting 
MDS  soiutions  for  the  individual  data  sots  had  low  stress  values,  and  correlations  between 
common-item  IPDs  from  independently  derived  data  set.s  were  relatively  high.  These  results 
indicated  tliat  the  goodness  of  fit  between  the  distance  estinriates  derived  from  these  solutions 
and  the  original  similarity  rating  averages  was  good  to  excellent  (according  to  goodness-of-fit 
classifications  by  Kruskal,  1964),  and  that  the  distance  v.aiues  were  highly  reliable  in  terms  of 
replication.  Based  on  these  findings,  it  was  concluded  that  the  validity  of  the  distance  measures 
derived  from  these  solutions  was  highly  satisfactory. 

Although  distance  data  derived  from  MDS  solutions  of  individuai  data  sets  were  deemed 
acceptable  in  terms  of  validity,  they  did  not  provide  distances  from  targets  to  cross-category 
alternatives.  It  should  be  reiterated  that  the  original  objective  of  the  scaling  work  was  to  derive 
distance  measures  atnong  all  possible  pairwise  combinations  of  menu  stimuli  so  that  an  optimal 
measure  for  Di  could  bn  empirically  determined.  Without  distance  measures  between  targets 
and  oitornatives  from  other  categories,  the  operational  definition  of  the  selection  competition 
measure  (Di)  was  limiteri  in  Include  only  estimates  of  the  distances  from  the  correct  menu 
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alternative  to  incorrect  ones.  Should  the  results  of  this  experiment  confirm  hypothesized 
performance  effects,  the  impact  of  the  limited  Di  measure  would  simply  be  the  problem  of 
generalizing  the  results  to  broader  definitions  of  menu  set  discriminability  produced  by  competing 
alternatives.  On  the  other  hand,  should  the  results  show  no  significant  effects  for  the  Di  factor, 
it  would  not  be  possible  to  distinguish  whether  these  results  were  due  to  the  true  absence  of 
a  main  effect  or,  rather,  due  to  an  inadequate  level  of  measurement  scnsitivity/reliahility  associated 
with  the  limited  Di  measure. 

In  summary,  even  without  distance  estimates  for  targets  to  cross-category  items,  the  scaling 
procedures  did  achieve  the  objective  of  providing  a  set  of  semantic  proximity  measures  to  be 
used  to  quantify  menu  item  relationships  in  the  menu  task  experiment.  These  data  provided 
the  means  to  evaluate  semantic  manipulations  of  menus  and  the  effect  these  manipulations  had 
on  task  performances. 


General  Approach 

A  partial  search  task  procedure  was  used.  Subjects  attempted  to  match  a  target,  presented 
at  the  onset  of  each  trial,  with  the  correct  alternative  contained  in  a  2-,  4-  or  8-alternative  menu, 
which  was  displayed  following  target  presentation.  Following  examination  of  the  menu  display, 
subjects  made  a  response  from  options  available  for  that  menu  page.  If  the  response  resulted 
in  an  error,  the  subject  continued  with  that  trial  until  the  correct  response  was  made.  Both 
simultaneous  and  sequential  modes  of  presenting  alternatives  were  used.  In  the  simultaneous 
mode,  all  alternatives  were  displayed  vertically  at  the  same  time.  In  the  sequential  mode, 
alternatives  were  displayed  one  at  a  time  in  a  sequential,  top-down  manner.  Rate  of  presentation 
in  this  latter  condition  was  controlled  by  the  subject.  The  simultaneous  presentation  mode 
provided  data  that  were  used  only  to  determine  accuracy  of  subject  responses.  The  sequential 
presentation  condition  provided  these  measures,  as  well  as  a  direct  measure  of  search  strategy 
for  individual  trial  responses.  The  combination  of  sequential  and  simultaneous  procedures  allowed 
the  study  of  decision  and  search  processes  employed  by  the  subjects,  as  well  as  permitting 
the  assessment  of  whether  the  forced  sequential  search  imposed  on  subjects  in  the  sequential 
condition  influenced  response  accuracy. 


Experimental  Design 

A  mixed  experimental  design  was  used,  with  two  between-groups  factors  and  three  within-group 
factors  (see  Table  4).  The  first  between-groups  factor  was  the  Presentation  Mode-either  sequential 
or  simultaneous-used  for  display  of  alternatives.  The  second  between-groups  factor,  the  Omission 
Probability  factor,  represented  four  levels  of  omission  trial  probability  (i.e.,  0%,  12.5%,  25%,  and 
37.5%).  The  first  within-group  factor  was  the  Number  of  Alternatives  factor,  representing  the 
number  of  alternatives  presented  on  the  menu  page  (either  2,  4,  or  8).  The  second  within 
factor  was  the  Hierarchy  factor,  representing  the  two  levels  in  the  menu  hierarchy  from  which 
each  menu  was  derived.  These  levels  were  defined  by  the  hierarchical  distance  between  the 
target  and  the  correct  alternative  in  the  menu.  In  one  level  of  the  Hierarchy  factor,  menus  were 
constructed  using  alternatives  from  Level  III  of  the  Figure  1  hierarchy.  In  the  other  level,  menus 
were  constructed  using  alternatives  from  Level  V.  The  third  within  factor  was  two  levels  of  the 
Nesting  factor,  representing  that  level  in  the  hierarchy  which  contained  a  superordinate  common 
to  all  alternatives  in  the  menu  display.  Thus,  these  levels  were  defined  by  the  hierarchical 
distance  between  correct  and  incorrect  alternatives  in  the  menu.  In  the  Nesting  factor  close 
condition,  menus  were  constructed  such  that  the  relationship  between  the  correct  alternative 
and  all  incorrect  alternatives  in  the  menu  was  as  hierarchically  close  as  possible.  In  the  Nesting 
factor  distant  condition,  menus  were  constructed  such  that  the  hierarchical  relationship  between 
correct  and  incorrect  alternatives  was  relatively  distant.  The  Nesting  factor  was  manipulated 
only  within  the  4-alternative  menu  condition.  For  the  2-  and  8-alternative  menus,  the  Nesting 
factor  was  fixed  at  the  close  level. 


23 


Table  4.  Experimental  Design 


Between-groups  factors 

Within-group 

factors 

(12  subjects 

Hierarchy 

Hierarchy 

per  group) 

Level  III 

Level  V 

Number  of  alternatives 

Number  of  alternatives 

2  4  4  8 

2  4  4  8 

Presen- 

tafion  Omission 

Nesting 

Nesting 

Group  mode  probability  close  close  dist  close  close  close  dist  close 


1  Seq  0% 

2  12.5% 

3  25% 

4  37.5% _ 

5  Sim  0% 

6  12-5% 

7  25% 

8  37.5% 

Note.  Seq  ■  sequential  presentation  mode,  Sim  -  simultaneous  presentation  mode. 


Apparatus 

IBM  PC-compatible  computers  with  12-inch  green  monochrome  monitors  were  employed  to 
present  stimuli,  and  record  responses. 


Subjects 

The  subjects  were  undergraduate  college  students  enrolled  in  an  introductory  psychology 
course  at  Arizona  State  University  The  design  required  eight  groups,  with  twelve  subjects  per 
group 


Procedure 


Each  subject  performed  two  practice  trials,  followed  by  128  experimental  trials.  The  2-alternative 
menu  trials,  4-alternative  Nesting  distant  trials,  4-alternative  Nesting-close  trials,  and  8-alternative 
trials  were  all  grouped  across  the  two  Hierarchy  factor  levels.  This  resulted  in  four  trial  sets, 
with  32  trials  per  set.  Withir.-group  ordering  effects  were  controlled  by  counterbalancing  the 
sequencing  of  trial  sets  across  subjects.  All  subjects  within  each  group  performed  identical 
sets  of  trials  using  the  same  target-to-menu-condition  pairings,  positioning  of  menu  alternatives 
within  trials,  and  omission  trial  assignments  Presentation  order  of  trial  sets  associated  with  the 
within-group  factors  was  counterbalanced  across  subjects.  Order  of  trial  presentation  within  trial 
sets  was  randomized  across  the  first  group  of  subjects.  Subjects  in  both  the  sequential  and 
simultaneous  conditions,  as  well  as  the  Omission  Probability  conditions,  were  then  yoked  with 
respect  to  trial  presentation  order.  Thus,  the  only  procedural  difference  between  subject  groups 
was  the  stimulus  Presentation  Mode  crossed  by  Omission  Probability;  the  only  procedural 
difference  among  subjects  within  groups  was  the  presentation  order  of  trials. 
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Each  trial  set  began  with  presentation  of  a  message  which  described  the  number  of  trials 
to  be  completed  (32  for  each  set)  and  the  Number  of  Alternatives  contained  in  each  menu. 
Individual  trials  began  with  presentation  of  the  target.  Subjects  initiated  presentation  of  alternatives 
by  pressing  the  space  bar. 

Procedures  similar  to  those  employed  by  MacGregor  et  al.  (1986)  were  employed  for  both 
the  sequential  and  simultaneous  Presentation  Mode  conditions.  In  the  sequential  condition,  only 
the  identification  numbers  corresponding  to  each  alternative  were  initially  visible.  Subjects 
displayed  the  first  alternative  in  the  menu  by  striking  the  space  bar.  Subjects  controlled  cursor 
movement  on  the  screen  by  pressing  the  "up"  and  "down"  special  function  keys.  Subjects 
displayed  alternatives  by  moving  the  cursor  to  its  corresponding  position  on  the  screen  and 
striking  the  space  bar.  Any  previously  displayed  alternative  then  was  erased  from  the  screen 
and  the  newly  selected  alternative  was  displayed  in  its  own  position.  Subjects  could  display 
the  next  alternative  in  the  menu  sequence  or  any  previously  viewed  alternatives:  however,  they 
could  not  skip  ahead  in  the  menu  over  alternatives  not  viewed.  In  the  simultaneous  condition, 
all  alternatives  were  displayed  on  the  screen  at  the  same  time.  In  all  conditions,  the  alternatives 
on  each  page  were  arranged  in  a  column,  with  one  alternative  per  line. 

In  both  simultaneous  and  sequential  conditions,  subjects  made  their  choice  by  pressing  the 
A"  key,  followed  by  the  number  of  the  alternative  selected  (or  zero  if  the  subject  believed  that 
the  target  was  not  to  be  found  under  any  of  the  alternatives  on  the  page).  In  the  sequential 
condition,  only  the  alternatives  viewed  were  available  for  selection.  The  zero  response  was 
allowed  only  after  all  menu  alternatives  were  viewed.  Thus,  guessing  was  not  permitted.  A 
correct  response  in  either  condition  led  to  presentation  of  the  next  search  target.  An  incorrect 
or  unacceptable  response  led  to  an  error  message  and  a  request  to  try  again.  In  the  sequential 
condition,  unacceptable  responses  indicative  of  guessing  were  not  treated  as  errors  but  rather, 
as  an  inappropriate  key  response.  In  this  situation,  a  message  would  be  displayed  which 
informed  the  subject  of  the  problem  and  requested  that  the  subject  continue  the  trial.  Subjects 
continued  each  trial  until  the  correct  response  was  made.  At  any  time,  a  subject  could  redisplay 
the  target  by  pressing  the  "T"  key. 


Materials 


Menu  Construction 


Assumptions.  The  semantically  scaled  menu  set  items  were  used  in  the  current  experiment 
to  form  the  45  menu  pages  depicted  in  Tables  5  and  6.  Manipulations  of  the  semantic 
relationships  among  targets  and  menu  alternatives  were  based  on  two  assumptions  derived  from 
semantic  memory  models  described  in  the  literature  (e  g.,  Collins  &  Quillian,  1969;  Rips  et  el., 
1973).  The  first  assumption  was  that  the  semantic  distance  between  a  target  and  a  correct 
alternative  is  directly  related  to  the  hierarchical  distance  between  them.  Thus,  as  the  hierarchical 
distance  increases,  the  semantic  distance  will  correspondingly  increase.  Based  on  this  assumption, 
the  distances  between  targets  and  correct  alternatives  (the  Hierarchy  factor)  were  manipulated 
by  pairing  targets  (Level  VI  items  in  Figure  1)  with  alternatives  from  either  Level  III  or  Level  V 
of  the  Figure  1  hierarchy. 

The  second  assumption  on  which  semantic  distance  manipulations  were  based  was  that 
menu  alternatives  sharing  a  common  superordinate  at  the  same  level  in  the  menu  hierarchy 
were  semantically  closer  to  each  other  than  to  alternatives  sharing  common  origins  at  higher 
levels  in  the  hierarchy.  For  example,  the  categories  "Flower,"  "Vegetable,"  "Grain,"  and  "Grazing" 
all  stem  from  the  same  Level  III  superordinate  "Agriculture"  in  Miller’s  (1981)  hierarchy  (see 
Figure  1).  "Flower"  and  "Vegetable,"  however,  also  share  the  hierarchically  closer  Level  IV  item 
"Garden."  Because  neither  "Grain"  nor  "Grazing"  is  subordinate  to  "Garden,"  the  assumption 
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argues  that  “Flower”  and  “Vegetable"  should  be  closer  semantically  to  one  another  than  either 
item  is  to  “Grain”  or  “Grazing.”  Based  on  this  assumption,  the  distances  among  the  menu 
alternatives  (the  Nesting  factor)  were  manipulated  by  grouping  alternatives  based  on  their 
superordinate  origins  at  various  levels  in  Miller's  (1981)  original  hierarchy.  With  the  exception 
of  menus  contained  in  the  Nesting  distant  condition  (see  Table  4),  all  menu  alternatives  were 
grouped  as  hierarchically  close  as  possible.  It  should  be  noted  that  these  latter  menus  were 
arranged  as  th'^y  would  have  appeared  had  the  user  actually  navigated  to  that  menu  level  from 
superordinate  levels  in  the  hierarchy. 

Table  5.  Menus  Constructed  from  Level  III  of  the  Hierarchy  (9  menus  total) 


2-Alternative  Menus  (4  menus) 


Menu  #1 

Agriculture 

Physics 

4-Alternative  Menus 

Nesting  Close  (2  menus) 


Menu  #2 

Medicine 

Zoology 


Menu  #1 

Agriculture 

Physics 

Medicine 

Zoology 


Menu  #2 

Country 

Topography 

Art 

Person 


8-Alternative  Menu  (1  menu) 

Agriculture 

Physics 

Medicine 

Zoology 

Country 

Topography 

Art 

Person 


Menu  #3 

Country 

Topography 


Menu  #4 
Art 

Person 


Nesting  Distant  (2  menus) 

Menu  #1  Menu  #2 


Agriculture 

Medicine 

Country 

Art 


Physics 

Zoology 

Topography 

Person 


Menu  Page  Preparation.  Based  on  the  assumptions  and  procedures  outlined  above,  menus 
for  trials  in  wh>ch  a  distant  semantic  relationship  was  desired  between  the  target  and  correct 
alternative  were  prepared  using  Level  III  alternatives  in  the  hierarchy  depicted  in  Figure  1.  Menus 
for  trials  in  which  a  close  semantic  relationship  was  desired  between  the  target  and  correct 
alternative  contained  only  Level  V  alternatives.  Nine  menu  pages  using  Level  III  alternatives 
were  generated  (see  Table  5):  Four  pages  contained  two  closely  related  alternatives;  two  pages 
contained  four  closely  related  alternatives;  two  pages  contained  four  distantly  related  alternatives; 
and  one  page  contained  eight  closely  related  alternatives.  Thirty-six  menu  pages  were  developed 
using  alternatives  selected  from  Level  V  of  the  hierarchy  (review  Table  6):  Sixteen  pages 
contained  two  closely  related  alternatives;  eight  pages  contained  four  closely  related  alternatives; 
eight  pages  contained  four  distantly  related  alternatives;  and  four  pages  contained  eight  closely 
related  alternatives. 

To  minimize  the  semantic  distance  among  the  four  alternatives  used  in  menus  constructed 
for  the  Hierarchy-Level  III  x  Nesting-close  x  4-Alternatives  condition,  alternatives  from  Level  III 
of  the  hierarchy  stemming  from  a  common  supererdinate  in  Level  I  of  Figure  1  were  grouped 
together.  Menus  constructed  for  the  Hierarchy-Level  V  x  Nesting-close  x  4-Alternatives  condition 
were  designed  by  grouping  alternatives  from  Level  V  of  the  hierarchy  which  had  a  common 
superordinate  at  Level  III  of  the  hierarchy.  To  maximize  the  semantic  distance  among  the  4 
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Table  6.  Menus  Constructed  from  Level  V  of  the  Hierarchy  (36  menus  total) 


2-Alternative  Menus  (16  menus) 


Menu  #1 

Menu  #2 

Menu  #3 

Menu  #4 

Flower 

Grain 

Force 

Solid 

Vegetable 

Grazing 

Heat 

Liquid 

Menu  #5 

Menu  #6 

Menu  #7 

Menu  #8 

Skeletal 

Child  Illness 

Freshwater  Fish 

Mammal 

Organ 

Adult  Illness 

Saltwater  Fish 

Reptile 

Menu  #9 

Menu  #10 

Menu  #11 

Menu  #12 

Eastern  Europe 

Northern  Asia 

Desert 

Ocean 

Western  Europe 

Southern  Asia 

Mountain 

Lake 

Menu  #13 

Menu  #14 

Menu  #15 

Menu  #16 

Music 

Graphics 

President 

Ruler 

Dance 

Sculpture 

General 

Philosopher 

-Alternative  Menus 

Nesting  Close  (8  menus) 

Menu  #1 

Menu  #2 

Menu  #3 

Menu  #4 

Flower 

Freshwater  Fish 

Skeletal 

Force 

Vegetable 

Saltwater  Fish 

Organ 

Heat 

Grain 

Mammal 

Child  Illness 

Solid 

Grazing 

Reptile 

Adult  Illness 

Liquid 

Menu  #5 

Menu  #6 

Menu  #7 

Menu  #8 

Eastern  Europe 

Desert 

Music 

President 

Western  Europe 

Mountain 

Dance 

General 

Northern  Asia 

Ocean 

Graphics 

Ruler 

Southern  Asia 

Lake 

Sculpture 

Philosopher 

Nesting  Distant  (8  menus) 

Menu  #1 

Menu  #2 

Menu  #3 

Menu  #4 

Flower 

Vegetable 

Grain 

Solid 

Skeletal 

Organ 

Child  Illness 

Mammal 

Eastern  Europe 

Western  Europe 

Northern  Asia 

Ocean 

Music 

Dance 

Graphics 

Ruler 

Menu  #5 

Menu  #6 

Menu  #7 

Menu  #8 

Force 

Heat 

Grazing 

Liquid 

Freshwater  Fish 

Saltwater  Fish 

Adult  Illness 

Reptile 

Desert 

Mountain 

Southern  Asia 

Lake 

President 

General 

Sculpture 

Philosopher 
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Table  6  (Concluded) 


8-Alternative  Menus  (4  menus) 


Menu  #1 

Menu  #2 

Menu  #3 

Menu  #4 

Music 

Eastern  Europe 

Skeletal 

Flower 

Dance 

Western  Europe 

Organ 

Vegetable 

Graphics 

Northern  Asia 

Child  Illness 

Grain 

Sculpture 

Southern  Asia 

Adult  Illness 

Grazing 

President 

Desert 

Freshwater 

Fish  Force 

General 

Mountain 

Saltwater 

Fish  Heat 

Ruler 

Ocean 

Mammal 

Solid 

Philosopher 

Lake 

Reptile 

Liquid 

alternatives  used  in  the  Hierarchy-Level  III  x  Nesting-distant  x  4-Alternatives  condition,  alternatives 
from  Level  III  in  the  hierarchy  stemming  from  separate  superordinates  at  Level  II  of  the  hierarchy 
were  used  to  form  each  menu.  Similarly,  menus  in  the  Hierarchy-Level  V  x  Nesting-distant  x 
4-Alternatives  condition  were  created  by  selecting  alternatives  from  Level  V  in  the  hierarchy 
having  common  Level  1  origins  yet  stemming  from  separate  superordinates  at  Level  III  of  the 
hierarchy. 


Target-to-Menu  Pairings 

The  64  Level  VI  targets  were  divided  into  16  groups  consisting  of  four  targets  each  such 
that  all  targets  in  each  group  shared  a  common  Level  IV  superordinate  (see  Figure  1).  Targets 
in  each  group  were  then  assigned  for  use  with  each  of  the  four  conditions  defined  by  levels 
of  the  Nesting  and  Alternatives  factors.  This  procedure  was  done  separately  for  each  of  the 
two  levels  of  the  Hierarchy  factor.  For  each  of  the  16  target  groups,  assignment  of  targets  to 
menu  conditions  was  made  using  a  random  selection  technique  (with  the  restriction  that  only 
one  target  from  each  group  could  be  assigned  to  each  of  the  four  menu  conditions).  Once 
target-to-menu-condition  assignments  had  been  completed,  the  menu  page  containing  the  correct 
alternative  for  the  target  assigned  to  that  condition  was  then  paired  with  the  target  stimulus. 
Within  each  of  the  two  levels  of  the  Hierarchy  factor,  individual  targets  were  used  only  once. 
Each  target  therefore  was  used  twice  across  both  levels  of  this  factor.  As  a  result  of  this 
procedure,  each  subject  performed  a  total  of  128  trials.  Ordering  of  alternatives  was  randomized 
for  each  menu  display  (with  the  restriction  that  the  location  of  the  correct  alternative  on  each 
menu  display  was  distributed  equally  across  trials  over  the  available  positions). 


Verifying  Assumptions 


Problem  Issues 


The  intent  of  the  Hierarchy  manipulation  was  to  produce  two  levels  of  menu  stimuli  which 
differed  significantly  with  respect  to  the  semantic  relationship  between  the  target  and  the  correct 
alternative.  Similarly,  the  intent  of  the  Nesting  manipulation  was  to  produce  two  levels  of  menu 
stimuli  which  significantly  differed  with  respect  to  the  semantic  relationship  between  correct  and 
incorrect  alternatives.  The  intent  of  the  Alternatives  manipulation  was  simply  to  create  menus 
in  which  the  Number  of  Alternatives  presented  for  selection  varied  from  level  to  level.  Although 
the  intent  of  the  Alternatives  manipulation  was  readily  verified  by  counting  Number  of  Alternatives, 
the  variation  of  semantic  relationships  resulting  for  the  manipulations  associated  with  the  Hierarchy 
and  Nesting  factors  required  a  more  sophisticated  examination. 
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Another  issue  requiring  resolution  concerned  the  potential  confounding  effects  of  manipulations 
on  semantic  relationships  among  stimuli.  Specifically,  the  potential  effects  of  variations  in  the 
Alternatives  and  Hierarchy  factors  on  the  semantic  relationships  between  correct  and  incorrect 
alternatives  needed  to  be  assessed.  Similarly,  the  potential  effects  of  variations  in  the  Alternatives 
and  Nesting  factors  on  the  semantic  relationships  between  targets  and  correct  alternatives  used 
in  the  menus  needed  to  be  examined. 


Analytic  Procedures 

Semantic  proximity  manipulations  for  the  menus  and  trial  arrangements  defined  above  were 
tested  statistically  through  a  series  of  ANOVAs  using  as  dependent  variables  the  distance  values 
estimated  from  the  previously  described  scaling  procedures.  Two  sets  of  ANOVAs  were  computed. 
In  the  first  set,  the  effects  due  to  manipulations  of  the  Alternatives  and  Hierarchy  factors  were 
analyzed  using  the  following  three  dependent  measures  in  separate  analyses:  (a)  scaled  distances 
between  targets  and  correct  menu  alternative  (scaled  distance  to  correct  alternative  -  Dc),  (b) 
scaled  distance  between  the  correct  alternative  and  the  semantically  nearest  incorrect  alternative 
in  the  menu  (scaled  distance  to  nearest  neighbor  -  Di-NN),  and  (c)  the  average  of  the  scaled 
distances  between  the  correct  alternative  and  all  incorrect  alternatives  in  the  menu  (Di-AVE). 
The  second  set  of  ANOVAs  tested  the  effects  that  manipulations  of  the  Nesting  and  Hierarchy 
factors  had  on  the  same  dependent  variables  examined  in  the  first  ANOVA  set.  These  procedures 
not  only  allowed  the  assessment  of  the  intent  of  the  menu  manipulations,  but  also  allowed  the 
assessment  of  potential  confounding  among  the  design  factors  with  respect  to  Dc  and  Di 
measures.  What  follows  is  a  summary  of  the  procedures,  findings  and  implications  of  these 
ANOVAs.  2 


Findings  and  Implications 

The  analyses  of  manipulation  effects  on  scaled  distance  values  have  several  implications 
concerning  the  effects  of  the  factor  manipulations  and  the  type  of  analytic  procedures  that 
should  be  used  when  interpreting  menu  task  performance.  First,  findings  involving  the  Hierarchy 
and  Nesting  factors  were  consistent  with  respect  to  the  expected  effects  of  these  factors  on 
the  semantic  relationships  among  targets  and  alternatives.  Level  III  menus  consistently  had 
significantly  greater  mean  Dc  values  than  did  Level  V  menus.  Second,  the  distant  level  of  the 
Nesting  factor  consistently  had  significantly  greater  mean  Di-NN  and  Di-AVE  values  than  did  the 
close  level. 

Of  equal  importance  was  the  finding  that  the  main  effects  on  mean  Di-NN  and  Di-AVE 
measures  for  both  the  Alternatives  and  the  Hierarchy  factors  were  consistently  significant  in  the 
first  set  of  ANOVAs.  In  addition,  the  Hierarchy  by  Nesting  interaction  on  mean  Di  measures 
was  consistently  significant  in  the  second  set  of  ANOVAs.  These  findings  suggest  a  confounding 
of  both  the  Alternatives  and  the  Hierarchy  factors  due  to  the  effect  these  factors  have  on  the 
semantic  relationship  between  correct  and  incorrect  alternatives. 

The  confounding  among  these  factors  implied  that  in  order  to  assess  the  effects  of  these 
variables  on  menu  task  performance,  a  statistical  technique  was  required  that  would  allow  the 
removal  of  unwanted  sources  of  variance  for  each  variable  analyzed.  Regression  analysis 
provided  the  capability  to  partial  out  shared  variance  for  each  predictor  with  respect  to  all  other 


2A  detailed  summary  of  the  procedures  and  findings  of  the  menu  manipulation  ANOVAs  appears  in  Pierce  (1989). 
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predictors  contained  in  the  model.  Significance  testing  of  the  partial  coefficients  then  permitted 
the  required  assessment  of  the  unique  amount  of  variance  accounted  for  by  each  variable  in 
the  regression  model.  The  Dc,  Di-NN  and  Di-AVE  continuous  measures  for  each  trial  were  used 
in  place  of  the  levels  associated  with  the  Hierarchy  and  Nesting  factors.  Using  these  variables, 
a  regression  analysis  of  performance  data  which  removed  unwanted  variance  was  conducted. 
Section  IV  gives  a  complete  description  of  this  procedure. 

Omission  Trials 


Omission  trial  Probability  was  established  in  each  of  the  2-  and  4-alternative  menu  sets  by 
replacing  the  correct  alternatives  with  incorrect  ones.  The  number  of  omission  trials  presented 
in  a  given  block  of  trials  was  set  at  the  between-groups  condition  rate  for  each  group  of  subjects 
(Omission  Probability  factor  in  Table  4).  The  four  probability  levels  used  were  typical  of  error 
rates  obtained  in  previous  menu  navigation  studies  (e  g.;  MacGregor  et  al.,  1986;  Snowberry  et 
al.,  1983).  Omission  trials  for  the  8-alternative  menus  were  arranged  somewhat  differently. 
Because  the  8-alternative  menus  in  the  Hierarchy  Level  III  condition  contained  all  possible 
alternatives  from  that  level,  only  one  menu  was  constructed  for  the  Hierarchy  Level  III  x  8 
Alternatives  condition  (see  Table  5).  Thus,  replacement  of  the  correct  Level  III  alternative  with 
an  incorrect  one  from  the  menu  set  was  not  possible.  Therefore,  all  Omission  trials  in  the 
8-alternative  trials  were  constructed  using  Hierarchy  Level  V  menus. 

Construction  of  Omission  trials  proceeded  as  follows.  Semantic  distances  between  the  correct 
menu  alternative  and  its  semantically  closest  neighbor  that  was  not  a  member  of  the  menu  in 
question  were  calculated  for  each  trial.  For  each  Hierarchy  Level  III  menu,  only  Level  III 
alternatives  were  considered  to  be  neighbors.  In  a  similar  manner,  only  Level  V  alternatives 
were  considered  for  Hierarchy  Level  V  menus.  Within  each  trial  condition,  trials  having  the 
closest  distances  between  the  correct  alternative  and  its  nearest  available  neighbor  were  selected 
to  be  Omission  trials.  For  those  trials  so  selected,  the  correct  alternative  was  replaced  with 
its  nearest  available  neighbor.  In  trial  sets  where  the  same  correct  alternative  was  used  more 
than  one  time  with  the  same  menu,  once  selected  the  alternative  was  not  considered  again  until 
all  other  available  alternatives  had  been  used  an  equal  number  of  times. 

The  above  procedures  were  intended  to  provide  Omission  trial  settings  that  were  not  highly 
obvious  to  the  subject.  The  use  of  these  procedures,  however,  may  have  introduced  two 
potential  sources  of  confounding  effects.  First,  by  selecting  the  semantically  nearest  available 
incorrect  alternative  as  a  replacement,  Di-NN  and  Di-AVE  values  may  have  dec.i-eased  with 
increasing  levels  of  the  Omission  factor.  The  second  potential  confounding  source  concerns 
the  effect  the  Omission  trial  selection  procedure  had  on  Dc  values.  Omission  trials  were  formed 
using  those  trials  having  the  closest  distances  between  the  correct  alternative  and  its  nearest 
available  neighbor;  therefore,  to  the  extent  that  variations  in  Dc  and  Di-NN  measures  may  have 
been  correlated,  the  possibility  exists  that  mean  Dc  values  for  trials  in  which  the  Correct 
Alternative  was  Present  were  also  confounded  across  levels  of  the  Omission  Probability  factor. 

The  effect  that  Omission  trial  selection  procedures  had  on  Dc,  Di-NN  and  Di-AVE  values  was 
examined  analytically  by  computing  two  sets  of  ANOVAs,  using  an  approach  similar  to  the  one 
used  to  verify  menu  manipulations.  Two  sets,  each  consisting  of  three  ANOVAs,  were  computed. 
In  the  first  set,  the  effects  on  each  of  three  semantic  distance  measures  separately  were  tested 
for  manipulations  of  the  Alternatives,  Hierarchy,  and  Omission  factors,  along  with  a  fourth  factor 
which  identified  whether  or  not  the  Correct  Alternative  was  Present  in  the  menu.  The  Nesting 
factor  was  fixed  at  the  close  level  in  this  first  set  of  analyses.  In  the  second  set  of  ANOVAs, 
the  manipulations  of  the  Hierarchy,  Nesting,  Omission  and  Correct  Alternative  Present  factors 
were  tested  in  three  separate  ANOVAs  for  their  effects  on  Dc,  Di-NN  and  Di-AVE.  Thus,  Dc, 
Di-NN  and  Di-AVE  values  for  individual  trials  were  the  basic  unit  of  analysis  in  both  sets  of 
ANOVAs.  In  all  ANOVAs,  the  Dc  distance  measures  for  Correct  Alternative  Absent  trials  represented 
the  distance  from  the  target  to  the  correct  alternative  that  was  removed  from  the  menu  to  form 
the  Omission  trial. 
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None  of  the  tests  in  either  set  of  analyses  indicated  significant  main  effects  for  the  Omission 
factor.  In  addition,  no  interaction  terms  containing  the  Omission  factor  had  significant  effects 
on  any  of  the  dependent  variables.  However,  the  main  effect  for  the  factor  representing  the 
Presence  or  Absence  of  the  Correct  Alternative  was  significant  in  all  ANOVAs,  as  were  several 
interaction  terms  containing  this  factor.  Examination  of  Table  7  indicates  that  means  for  all 
three  distance  measures  were  greater  for  the  Correct  Alternative  Present  trials  than  they  were 
for  Correct  Alternative  Absent  trials.  The  results  suggest  that  the  Omission  trial  selection 
procedure  did  not  have  a  significant  effect  on  the  relationships  among  targets  and  alternatives 
across  levels  of  the  Omission  factor.  However,  the  results  also  suggest  that  a  stronger  relationship 
exists  between  targets  and  incorrect  menu  alternatives  when  the  Correct  Alternative  is  Absent 
than  when  the  Correct  Alternative  is  Present. 

Table  7.  Comparison  of  Mean  Do,  Di-NN  and  Di-AVE  Values  for  Correct 
Alternative  Present  versus  Absent  Trials 


Correct  alternative 


Present 

Absent 

Fixed:  Nesting  close 

Dc  M 

.759 

.626 

n 

(312) 

(72) 

Di-NN  M 

.983 

.693 

n 

(312) 

(72) 

Di-AVE  M 

1.21 

1.02 

n 

(312) 

(72) 

Fixed;  Alternatives  =  4 

Dc  M 

.782 

.676 

n 

(208) 

(48) 

Di-NN  M 

1.143 

.714 

n 

(208) 

(48) 

Di-AVE  M 

1.402 

1.162 

n 

(208) 

(48) 

Note.  Upper  means  for  all 

trials  where  Nesting  was  fixed  at  close: 

lower  means 

for  all  trials  where  Alternatives  were  fixed  at  4. 


The  results  of  these  analyses  suggest  that  separate  analyses  of  Correct  Alternative  Present 
data  and  Correct  Alternative  Absent  data  are  required.  These  results,  combined  with  the 
confounding  of  the  Alternatives  and  Hierarchy  factors  noted  earlier  from  analysis  of  menu  stimuli 
manipulations,  formed  the  basis  for  the  regression  procedures  used  during  the  analysis  of  menu 
task  performance  data. 


IV.  RESULTS 

The  performance  data  were  analyzed  in  two  phases.  The  first  phase  consisted  of  a  series 
of  regression  analyses  to  test  the  effects  of  Alternatives,  Omission  Probability,  Dc  values  and 
Di  values  on  response  accuracy.  An  additional  factor  examined  was  Presentation  Mode  (i.e., 
sequential  versus  simultaneous).  The  second  phase  focused  on  the  effects  of  Alternatives, 
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Omission  Probability,  Dc  values  and  Di  values  on  subjects’  search  strategies  using  sequential 
Presentation  Mode  data  only. 


Phase  I  -  Response  Accuracy 


Procedures 


Response  accuracy  data  analysis  procedures  followed  from  the  implications  given  by  the 
menu  manipulation  and  Omission  trial  selection  analyses  described  in  Section  III.  The  results 
of  these  analyses  suggested  that  performance  data  could  best  be  interpreted  using  a  regression 
approach.  Such  an  approach  would  allov.  the  analysis  of  the  Alternatives,  Omission  Probability, 
Dc,  and  Di  effects  by  computing  the  partial  coefficients  for  each  of  these  predictors  and  then 
testing  these  coefficients  for  significance.  The  results  of  the  Omission  trial  selection  analyses 
also  suggested  that  performances  on  Correct  Alternative  Present  trial?  should  be  analyzed 
separately  from  Correct  Alternative  Absent  trials.  Consequently,  data  associated  with  individual 
trials  were  separated  into  two  data  sets  as  defined  by  the  Presence  or  Absence  of  a  Correct 
Alternative.  Each  data  set  v/as  analyzed  separately. 

Correct  Alternative  Present  Trials.  Proportions  of  hit  responses  for  Correct  Alternative  Present 
trials  are  summarized  in  Table  8  for  individual  cells  of  the  experimental  design.  Regression 
analysis  of  hit  responses  proceeded  as  follows.  Proportions  of  hit  responses  for  individual  trials 
were  computed  across  subjects  for  each  of  the  eight  groups  and  used  as  the  basic  unit  of 
analysis.  Following  procedures  recommended  by  Cohen  and  Cohen  (1983)  for  transformation 
of  proportion  data,  the  arcsine  transformations  of  these  proportions  were  used  as  the  criterion 
variable.  Separate  regression  analyses  were  computed  for  the  sequential  Presentation  Mode 
groups,  the  simultaneous  Mode  groups,  and  the  combination  of  data  from  both  groups.  For 
each  data  set,  two  regression  analyses  were  computed.  Cne  used  the  Di-NN  value  and  the 
other  used  the  Di-AVE  value  as  the  predictor  variable  to  define  the  relationship  between  correct 
and  incorrect  alternatives.  In  all  regression  analyses,  additional  predictors  included  the  Number 
of  Alternatives,  emission  Probability  and  the  Dc  trial  value.  For  regression  analyses  in  which 
data  from  both  Presentation  Modes  were  combined,  a  predictor  variable  representing  Presentation 
Mode  was  added  to  the  above. 

Correci  Aiiernaiive  Absent  Trials 


Proportions  of  correct  rejections  tor  Correct  Alternative  Absent  trials  are  summarized  in  Tabla9 
for  individual  cells.  Procedures  sirr.'  ar  to  those  above  were  used  for  regression  analysis  of 
correct  rejection  data.  Proportion''  of  correct  rejection  responses  were  computed  across  subjects 
within  each  group  for  individual  trials  and  used  a^  the  basic  unit  of  analysis.  The  arcsine 
transformations  of  these  proportions  were  computed  and  used  as  the  criterion  variable.  Because 
all  trials  in  the  0%  emission  Probability  condition  contained  the  correct  alternative,  correct 
rejection  performance  data  from  only  six  groups  of  subjects  were  used.  As  with  the  Correct 
Alternative  Present  analyses,  separate  analyses  were  performed  on  data  from  each  Presentation 
Mode  in  addition  to  the  data  from  both  modes  combined.  Predictor  variables  were  identical  to 
those  used  in  the  analysis  of  Correct  Alternative  Present  response  accuracy  data. 


Findings 


Correct  Alternative  Present  Trials 


Table  10  summarizes  the  regression  analyses  results  for  Correct  Alternative  Present  trial 
performances  The  F-values  for  both  the  Di  NN  and  the  Di-AVE  predictor  models  were  significant 
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Marginal  Proportions  .852  .843  .837  .793  .914  .845  .910  .761  .845 

(across  all  groups) _ (1248)  (1248)  (1248)  (1536)  (1248)  (1248)  (1248)  (960) _ (9984) 

Note.  Number  of  observations  in  parentheses 


Table  9.  Cell-by-Cell  Proportions  of  Correct  Rejections  for  Correct  Alternative  Absent  Trials 
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lumber  of  observations  in  parentheses. 


Table  10.  Regression  Results  for  Proportion  of 
Hits  on  Correct  Alternative  Present  Trials 
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for  the  sequential  Mode  data,  the  simultaneous  Mode  data,  and  the  combined  Modes  data. 
R-square  values  show  the  cumulative  proportion  of  variance  accounted  for  with  the  addition  of 
the  predictor  variables  listed.  The  cumulative  R-squares  for  the  addition  of  the  Di-NN  and  Di-AVE 
predictors  indicate  that  both  variables  account  tor  approximately  tne  same  amount  of  variance 
for  all  three  data  sets  analyzed. 

Partial  correlations  and  coefficients  for  all  of  the  model  predictor  variables,  with  the  exception 
of  the  coefficient  for  the  Presentation  Mode  variable,  were  statistically  significant  in  all  analyses. 
The  tests  of  the  partial  correlations  and  coefficients  indicate  the  significance  of  the  unique 
variance  accounted  for  by  the  Alternatives,  Omission  Probability,  Dc,  and  Di  predictors. 

Comparison  of  the  partial  correlations  and  coefficients  derived  for  the  sequential  Presentation 
Mode  data  to  the  same  statistics  derived  for  the  simultaneous  Mode  data  indicates  a  high  degree 
of  similarity.  Partial  coefficients  resulting  from  the  regressions  for  these  two  data  sets  all  indicate 
the  same  positive  or  negative  relationship  to  the  criterion;  and,  for  the  most  part,  the  partial 
correlations  were  of  about  the  same  magnitude.  One  exception  was  the  partial  correlation 
derived  for  the  Omission  Probability  predictor.  Though  coefficients  for  this  variable  were  negative 
in  all  regressions,  the  Omission  Probability  partial  correlations  for  sequential  Mode  were  more 
than  triple  the  values  obtained  for  the  simultaneous  Mode.  This  finding  would  seem  to  indicate 
that  Omission  Probability  had  a  more  pronounced  effect  on  the  criterion  in  the  sequential 
Presentation  Mode  than  in  the  simultaneous  Presentation  Mode. 

The  correlations  and  coefficients  for  the  Presentation  Mode  predictor  were  not  significant  in 
either  regression  where  both  Modes  were  combined.  Additional  analyses  of  two-way  interaction 
terms  which  included  the  Presentation  Mode  variable  indicated  that  none  of  the  other  variables 
in  the  model  significantly  interacted  with  Presentation  Mode  (p  >  .05).  Thus,  the  results  suggest 
that  Presentation  Mode  did  not  have  a  significant  main  effect,  nor  did  it  significantly  moderate 
the  effects  of  other  predictor  variables.  These  results  are  consistent  with  the  hypothesis  that 
similar  decision-making  processes  are  used  in  both  Modes  and  that  the  Mode  factor  does  not 
have  a  significant  effect  on  decision  accuracy. 

Examination  of  the  significant  partial  coefficients  for  model  predictors  indicates  the  following 
relationships  were  consistent  across  all  three  data  sets:  (a)  As  the  Number  of  Alternatives, 
Omission  Probability  or  semantic  distance  between  target  and  correct  alternative  increased, 
proportion  of  hits  decreased;  (b)  conversely,  as  the  semantic  distance  (as  defined  by  either 
Di-NN  or  Di-AVE)  between  correct  and  incorrect  alternatives  increased,  the  proportion  of  hits 
increased.  These  results  are  consistent  with  the  hypothesis  that  response  accuracy  on  Correct 
Alternative  Present  trials  is  significantly  affected  by  variations  in  these  four  menu  design  factors. 


Correct  Alternative  Absent  Trials 


Results  of  the  regression  analyses  for  Correct  Alternative  Absent  data  are  presented  in 
Table  11.  F-values  for  the  regressions  were  significant  in  each  of  the  six  analyses  computed 
for  these  data.  R-square  values  show  the  cumulative  proportion  of  variance  accounted  for  with 
the  addition  of  the  predictor  variables  listed.  In  contrast  to  the  cumulative  R-squares  obtained 
from  analysis  of  Correct  Alternative  Present  data,  the  cumulative  R-squares  obtained  for  the 
present  analyses  indicate  that  the  Di-NN  predictor  accounted  for  an  additional  5%  to  11%  of 
the  variance  beyond  that  accounted  for  by  the  Di-AVE  predictor  across  the  three  data  sets. 

Examination  of  the  R-square  values,  along  with  the  results  of  the  significance  tests  for  the 
partial  correlations  and  coefficients,  indicates  that  the  variance  accounted  for  by  these  models 
was  due  mainly  to  the  inclusion  of  the  Di  predictors.  In  the  Di-NN  models,  tne  only  significant 
partial  correlations  and  coefficients  were  for  the  Di-NN  predictor  across  all  three  data  sets.  As 
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Table  1 1 .  Regression  Results  lor  Proportion  of  Correct  Rejections 
on  Correct  Alternative  Absent  Trials 
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Di-NN  values  increased,  proportions  of  correct  rejections  increased.  In  the  Di-AVE  models, 
partial  correlations  and  coefficients  for  both  the  Di-AVE  predictor  and  the  Alternatives  predictor 
were  significant.  As  Di-AVE  values  increased,  proportions  of  correct  rejections  increased. 
Conversely,  as  the  Number  of  Alternatives  increased,  proportions  of  correct  rejections  decreased. 

As  with  the  analysis  of  the  Correct  Alternative  Present  response  accuracy  data,  the  comparison 
of  significant  partial  correlations  and  coefficients  across  Presentation  Modes  indicates  a  high 
degree  of  similarity.  Significant  coefficients  all  have  the  same  sign  and  partial  correlations  all 
have  similar  values.  The  partial  correlations  and  coefficients  derived  for  the  Presentation  Mode 
predictor  were  not  significant  in  either  analysis  where  data  from  Presentation  Modes  were 
combined.  Further  testing  of  two-way  interaction  terms  which  included  the  Presentation  Mode 
variable  indicated  that  none  of  the  other  predictors  in  the  model  significantly  interacted  with 
Presentation  Mode  (p  >  .05).  These  results  complement  those  obtained  from  analysis  of  the 
Correct  Alternative  Present  data,  suggesting  that  the  Presentation  Mode  did  not  have  a  significant 
main  effect  on  response  accuracy,  nor  did  it  significantly  moderate  the  effects  of  other  model 
predictors. 


Performance  Consistency  Across  Presentation  Modes 

Regression  results  indicated  a  high  degree  of  consistency  between  subject  performances 
obtained  in  the  two  Presentation  Modes.  This  consistency  is  further  exemplified  in  Tables  12 
and  13.  These  tables  show  correlations,  means,  and  standard  deviations  of  response  accuracy 
measures  for  sequential  and  simultaneous  Mode  data  sets  within  cells  of  trials  as  defined  by 
the  Alternatives  and  Omission  factors.  Table  12  contains  these  values  for  arcsine  transformations 
of  the  proportion  of  hits  for  Correct  Alternative  Present  trials.  Table  13  contains  values  for 
arcsine  transformations  of  the  proportion  of  correct  rejections  for  Correct  Alternative  Absent 
trials.  The  cveral!  correlation  between  Presentation  Modes  for  arcsine  transformations  of  the 
proportions  of  hits  was  .674  (£  <  .001).  For  arcsine  transformations  of  the  proportions  of 
correct  rejections,  the  overall  correlation  was  .855  (£  <  .001).  The  correlation  results  complement 
the  findings  from  the  response  accuracy  analyses,  suggesting  that  subjects'  response  performances 
from  the  two  Presentation  Modes  were  comparable. 


Phase  II  -  Search  Strategies 

Phase  II  analyzed  the  effects  of  performance  factors  on  the  search  strategies  employed  in 
the  sequential  Presentation  condition.  With  the  data  generated  from  this  experiment,  there  was 
no  direct  means  to  verify  the  generalization  of  search  strategy  findings  from  analysis  of  sequential 
mode  data  to  simultaneous  presentation  of  alternatives.  However,  the  preceding  analyses  of 
response  accuracy  data  are  consistent  with  the  hypothesis  that  such  a  generalization  is  appropriate. 


Procedures 


Search  strategy  regression  analyses  proceeded  as  follows.  For  every  trial  performed  in  the 
sequential  Mode  condition,  the  numbers  of  different  and  redundant  alternatives  examined  prior 
to  making  a  response  were  recorded.  Search  strategy  was  inferred  from  these  data  and  classified 
as  self-terminating,  exhaustive  or  redundant  using  the  following  operational  definitions. 
Self-terminating  searches  were  defined  as  those  wherein  the  subject  selected  an  alternative  prior 
to  examination  of  all  alternatives  in  the  menu.  Exhaustive  search  was  defined  as  one  in  which 
all  alternatives  in  the  menu  were  examined  once  and  only  once  prior  to  selection.  Redundant 
search  was  defined  as  a  search  in  which  all  menu  alternatives  were  examined  and  some 
alternatives  re-examined  prior  to  selection.  These  definitions  imply  that  in  situations  where  the 
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subject  examines  all  alternatives  once  and  only  once  and  then  chooses  the  alternative  occupying 
the  last  position  of  the  menu,  search  strategy  would  be  classified  as  exhaustive.  It  could  be 
argued,  however,  that  such  searches  could  be  classified  as  self-terminating  in  that  the  subject 
terminated  the  search  upon  examination  of  the  alternative  selected.  Therefore,  search  strategy 
for  these  situations  was  considered  indeterminate  and  its  use  in  analyses  was  subject  to  the 
procedures  detailed  below. 


Table  12.  Correlations  Between  Presentation  Modes  for  Arcsine 
Transformations  of  Proportions  of  Hits-Correct  Alternative  Present  Trials  Only 


Omission 

probabiiity 

Number 

of  alternatives 

2 

4 

8 

r 

— 

644*** 

r 

_ 

.734*** 

r 

.495** 

n 

= 

32 

n 

= 

64 

n 

32 

0% 

M  seq 

= 

2.851 

M  seq 

= 

2.821 

M  seq 

2.457 

SD  seq 

= 

.518 

SD  seq 

= 

.575 

SD  seq 

= 

.576 

M  sim 

= 

2.666 

M  sim 

= 

2.670 

M  sim 

= 

2.365 

SD  sim 

= 

.511 

SD  sim 

= 

.523 

SD  sim 

= 

.512 

r 

.512** 

r 

= 

.757*** 

r 

— 

.684*** 

n 

= 

28 

n 

= 

56 

n 

= 

28 

12.5% 

M  seq 

= 

2.486 

M  seq 

= 

2.513 

M  seq 

= 

2.255 

SD  seq 

= 

.524 

SD  seq 

= 

.614 

SD  seq 

= 

.629 

M  sim 

= 

2.777 

M  sim 

= 

2.618 

M  sim 

= 

2.335 

SD  sim 

= 

.452 

SD  sim 

= 

.568 

SD  sim 

= 

.722 

r 

= 

.619** 

r 

= 

.659*** 

r 

.688*** 

n 

= 

24 

n 

= 

48 

n 

= 

24 

25% 

M  seq 

= 

2.611 

M  seq 

= 

2.393 

M  seq 

= 

2.206 

SD  seq 

r= 

.525 

SD  seq 

= 

.614 

SD  seq 

= 

.608 

M  sim 

= 

2.565 

M  sim 

2.524 

M  sim 

= 

2.344 

SD  sim 

.521 

SD  sim 

= 

.522 

SD  sim 

= 

.610 

r 

= 

.678*** 

r 

= 

.684*** 

r 

.662** 

n 

= 

20 

n 

= 

40 

n 

= 

20 

37.5% 

M  seq 

= 

2.574 

M  seq 

= 

2.389 

M  seq 

= 

2.104 

SD  seq 

.613 

SD  seq 

= 

.640 

SD  seq 

= 

.788 

M  sim 

= 

2.515 

M  sim 

= 

2.466 

M  sim 

= 

2.199 

SD  sim 

= 

.714 

SD  sim 

= 

.692 

SD  sim 

= 

.741 

Overall: 

r 

= 

.674*** 

n 

= 

416 

M  seq 

2.508 

M  sim 

= 

2.533 

- - - ^ - - 

SD  seq 

= 

.630 

SD  sim 

= 

.594 

<  .01. 
*"p  <  .001. 
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Table  13.  Correlations  Between  Presentation  Modes  for  Arcsine 
Transformations  of  Proportions  of  Correct  Rejections -Correct  Alternative 

Absent  Trials  Only 


Omission 

Number  of  alternatives 

2 

4 

8 

probability 

r 

.982* 

r 

= 

.983*** 

r 

= 

.911 

n 

4 

n 

8 

n 

= 

4 

12.5% 

M  seq 

= 

1.283 

M  seq 

= 

2.071 

M  seq 

= 

1.959 

SD  seq 

1.366 

SD  seq 

= 

.953 

SD  seq 

= 

.171 

M  sim 

1.410 

M  sim 

2.083 

M  sim 

2.106 

SD  sim 

= 

.808 

SD  sim 

= 

1.007 

SD  sim 

.225 

r 

= 

.958*** 

r 

= 

.870*** 

r 

= 

.664 

n 

8 

n 

= 

16 

n 

= 

8 

25% 

M  seq 

= 

1.639 

M  seq 

z= 

1.870 

M  seq 

1.592 

SD  seq 

= 

.939 

SD  seq 

= 

.807 

SD  seq 

= 

.410 

M  sim 

= 

1.821 

M  sim 

2.156 

M  sim 

= 

1.689 

SD  sim 

.998 

SD  sim 

= 

.801 

SD  sim 

= 

.544 

r 

= 

.890*** 

r 

= 

.795*** 

r 

= 

.817** 

n 

= 

12 

n 

= 

24 

n 

= 

12 

37.5% 

M  seq 

= 

1.967 

M  seq 

= 

2.246 

M  seq 

= 

1.779 

SD  seq 

= 

.764 

SD  seq 

= 

.486 

SD  seq 

= 

.772 

M  sim 

= 

2.095 

M  sim 

= 

2.037 

M  sim 

= 

1.917 

SD  sim 

= 

.909 

SD  sim 

= 

.576 

SD  sim 

= 

.660 

Overall: 

r 

.855*** 

n 

= 

96 

M  seq 

= 

1.918 

M  sim 

= 

1.983 

SD  seq 

= 

.752 

SD  sim 

= 

.746 

*p  <  ,05. 

**p  <  .01, 

'**p  <  .001 


Three  sets  of  regression  analyses  were  computed.  Because  subjects  were  limited  to  employing 
only  exhaustive  and  redundant  searches  on  trials  where  the  Correct  Alternative  was  Absent 
(assuming  a  correct  response),  only  trials  in  which  the  Correct  Alternative  was  Present  were 
used  in  these  analyses.  In  the  first  set  of  regressions,  arcsine  transformations  of  the  proportion 
of  self-terminating  searches  were  computed  across  each  group  of  12  subjects  for  individual  trials. 
Only  trials  for  which  all  i2  subjects  exhibited  determinable  search  strategies  were  included  in 
the  analyses.  Two  regressions  were  performed  in  this  set.  The  first  included  the  Di-NN  variable 
as  the  predictor  representing  the  relationship  between  correct  and  incorrect  alternatives.  The 
second  included  the  Di-AVE  variable  as  the  predictor  representing  this  relationship.  In  both 
regressions,  Alternatives,  Omission  Probability  and  Dc  values  were  also  included  as  model 
predictors  with  the  arcsine  transforms  of  the  proportions  of  self-terminating  searches  serving  as 
the  criterion. 

In  the  second  set  of  regressions,  arcsine  transformations  of  the  proportion  of  exhaustive 
searches  for  individual  trials  were  used  as  the  criterion.  Proportions  of  exhaustive  searches 
were  computed  across  each  group  of  12  subjects  for  individual  trials.  Following  the  same 
procedure  used  for  self-terminating  search  data,  only  Correct  Alternative  Present  trials  for  which 
all  12  subjects  exhibited  determinable  search  strategies  were  included  in  the  analyses.  Two 
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regressions  were  computed,  one  using  Di-NN  as  the  predictor  representing  the  relationship 
between  the  correct  and  incorrect  alternatives,  and  the  other  using  Di-AVE  as  the  predictor.  In 
both  regressions,  additional  model  predictors  were  Alternatives,  Omission  Probability,  and  Dc 
values. 

In  the  third  regression  set,  arcsine  transformations  of  the  proportion  of  redundant  searches 
for  individual  trials  were  used  as  the  criterion.  Proportions  of  redundant  searches  were  computed 
across  each  group  of  12  subjects  for  individual  trials.  Because  computation  of  proportions  of 
redundant  searches  were  not  confounded  by  indeterminable  search  situations,  data  from  all 
Correct  Alternative  Present  trials  were  included  in  these  analyses.  In  addition,  to  be  consistent 
with  analyses  of  self-terminating  and  exhaustive  search  data,  regression  analyses  of  redundant 
searches  were  also  computed  for  Correct  Alternative  Present  trials  for  which  all  12  subjects 
exhibited  determinable  search  strategies.  As  with  the  previously  described  search  strategy 
regression  procedures,  two  regressions  were  computed  for  each  data  set,  using  Di-NN  as  a 
model  predictor  in  one  and  the  Di-AVE  variable  in  its  place  in  the  other.  In  all  regressions. 
Alternatives,  Omission  Probability  and  Dc  values  were  included  as  additional  model  predictors. 

Findings 

Proportions  of  self-terminating,  exhaustive  and  redundant  searches  for  Correct  Alternative 
Present  trials  for  which  all  12  subjects  exhibited  determinable  search  strategies  are  summarized 
in  Table  14  for  each  cell  included  under  the  sequential  Mode  condition.  Regression  results 
obtained  for  the  redundant  search  data  indicated  a  high  degree  of  consistency  between  analyses 
of  data  for  all  Correct  Alternative  Present  trials  versus  trials  on  which  all  12  subjects  exhibited 
determinable  search  strategies.  To  be  consistent  with  self-terminating  and  exhaustive  search 
analyses,  only  the  results  of  the  latter  analyses  of  redundant  searches  are  presented.  Table  15 
lists  the  results  of  the  six  search  strategy  regressions  computed  for  trials  where  all  12  subjects 
exhibited  determinable  searches.  F-tests  for  the  regression  models  were  significant  in  all  six 
analyses. 

Examination  of  the  partial  correlations  and  coefficients,  and  significance  test  results  for  the 
self-terminating  search  model  predictors  shows  that  all  predictors,  with  the  exce,Dtion  of  the  Di 
predictors,  significantly  contributed  to  the  model  regressions.  As  either  the  Number  of  Alternatives 
or  Omission  Probability  increased,  self-terminating  searches  increased.  As  the  semantic  distance 
between  targets  and  correct  alternatives  increased,  self-terminating  searches  decreased. 
Cumulative  R-square  values  listed  at  the  bottom  of  Table  15  indicate  that  both  the  Di-NN  and 
the  Di-AVE  models  accounted  for  approximately  37%  of  the  self-terminating  search  variance. 

The  results  from  analysis  of  exhaustive  search  data  indicate  that  only  the  Alternatives  and 
Dc  predictors  significantly  contributed  to  the  model  regressions.  Examination  of  the  partial 
coefficients  for  these  variables  indicated  that  as  the  Number  of  Alternatives  increased,  exhaustive 
searches  increased.  Cumulative  R-square  values  indicate  that  approximately  36%  of  the  variaiiue 
was  accounted  for  by  both  the  Di-NN  and  the  Di-AVE  models. 

All  four  partial  correlations  and  coefficients  in  the  redundant  search  models  significantly 
contributed  to  the  regressions  in  both  the  Di-NN  and  Di-AVE  models.  Examination  of  the 
coefficients  indicates  that  as  the  Number  of  Alternatives  and  as  the  semantic  distance  between 
target  and  the  correct  alternative  increased,  the  proportion  of  redundant  searches  increased. 
Redundant  searches  decreased  as  Omission  Probability  and  the  semantic  distance  from  correct 
to  incorrect  alternatives  increased.  Approximately  28%  of  the  variance  was  accounted  for  by 
both  the  Di-NN  and  Di-AVE  regression  models. 

In  all  three  sets  of  regressions  for  the  search  strategy  data,  the  comparison  of  the  Di-NN 
versus  Di-AVE  regression  models  indicated  little  difference  between  them  in  terms  of  significance 
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Table  14.  Cell-by-Cell  Proportions  of  Self-Terminating,  Exhaustive  and  Redundant  Searches 
for  Correct  Alternative  Present  Trials  Where  All  Subjects  Exhibited  Determinable  Search  Strategies 
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Marginal  Self-Terminating  .125  .139  .250  .342  .435  .375  .379  .489  .322 

Proportions  Exhaustive  .807  .708  .669  .482  .546  .544  .567  .368  .570 

(across  all  Redundant  .068  .153  .081  .176  .019  .081  .054  .142  .108 

groups)  (192)  (432)  (408)  (600)  (216)_  (408)  (480)  (372) _ (3,108) 

Note.  Number  of  observations  in  parentheses. 
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of  their  partial  correlations  and  coefficients  or  in  terms  of  variance  accounted  for  by  the  models. 
It  is  interesting  to  consider  the  relative  effects  of  the  Alternatives.  Omission  Probability,  Dc,  and 
Di  variables  with  respect  to  the  three  search  strategy  types  as  a  whole.  As  the  Number  of 
Alternatives  increased  in  the  menu,  the  proportion  of  both  self-terminating  and  redundant  searches 
tended  to  increase  at  the  expense  of  exhaustive  searches.  As  Omission  Probability  increased, 
the  proportion  of  self-terminating  searches  increased  at  the  expense  of  redundant  searches, 
although  this  variable  had  no  significant  effect  on  exhaustive  searches.  Increases  in  the  distance 
between  the  target  and  correct  alternative  increased  the  probability  of  both  exhaustive  and 
redundant  searches  at  the  expense  of  decreases  in  self-terminating  searches.  Lastly,  as  the 
distance  between  the  correct  alternative  and  incorrect  alternatives  increased,  redundant  searches 
decreased,  but  the  effect  of  this  variable  with  respect  to  either  self-terminating  or  exhaustive 
searches  was  not  significant.  The  overall  effects  demonstrated  through  these  analyses  for 
variations  in  the  Alternatives,  Omission  Probability,  Dc  and  Di  variables  support  the  hypothesis 
that  these  factors  significantly  influence  search  strategy  performance  on  computer  menu  tasks. 


Synopsis 

Analysis  of  search  strategy  and  response  accuracy  data  indicated  the  significant  effects  that 
Alternatives,  Omission  Probability,  and  Dc  and  Di  semantic  distance  measures  had  on  menu  task 
performance.  Furthermore,  analyses  of  response  accuracy  data  for  sequential  and  simultaneous 
Presentation  Modes  suggested  that  subjects’  decision  performances  from  the  two  Modes  were 
comparable.  This  latter  result  supports  the  generalization  of  search  strategy  findings  from  the 
sequential  Mode  to  simultaneous  presentation  of  alternatives. 

Although  the  analyses  reported  here  established  the  significance  of  menu  design  factors  with 
respect  to  task  performance,  their  implications  with  respect  to  cognitive  processes  involved  in 
menu  search  and  decision  making  are  limited.  The  next  section  provides  an  overview  of  a 
two-criterion  menu  model  which  suggests  a  process  that  can  potentially  account  for  the  effects 
of  menu  design  factors  on  task  performance.  The  next  section  also  details  the  results  of  a 
data-fitting  procedure  designed  to  assess  the  relationship  between  model  criteria  and  menu 
design  factors.  Further  discussion  of  the  results  of  the  menu  task  performance  analyses  and 
implications  for  menu  design  are  then  presented  in  Section  VI  in  the  context  of  the  two-criterion 
menu  model. 


V.  THE  TWO-CRITERION  MENU  MODEL 
Overview 

The  concept  of  a  two  criterion  menu  model  thai  couid  be  used  to  predict  menu  search 
strategies  and  response  outcomes  was  initially  proposed  in  Section  II.  The  genesis  of  this 
concept  was  the  Smith  et  al.  (1974)  feature  comparison  model  that  was  developed  to  explain 
cognitive  processes  involved  in  semantic  verification  tasks.  The  two-criterion  menu  model  extends 
concepts  proposed  by  Smith  et  al.  (1974)  to  the  multiple  verification  problem  involved  in  menu 
selection  tasks.  This  section  elaborates  the  development  of  the  two-criterion  menu  model  and 
details  procedures  used  to  fit  performance  data  obtained  from  the  menu  task  experiment  to 
model  prediction  functions.  Essentially,  performance  data  were  fitted  by  deriving  values  for  model 
parameters  that  minimized  differences  between  model  predictions  and  obtained  performances. 
The  derived  values  for  model  parameters  were  then  analyzed  through  a  series  of  regressions 
to  determine  the  relationship  between  model  criteria  and  menu  design  factors.  The  results  of 
these  procedures  were  used  to  explicate  the  cognitive  processes  involved  during  performance 
of  the  menu  search  task. 
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Description 


The  two-criterion  menu  model  was  designed  to  predict  combinations  of  search  strategy  and 
response  outcome  probabilities  under  a  variety  of  experimental  conditions  for  the  partial  search 
menu  task.  The  model  assumes  that  subjects  evaluate  alternatives  in  the  menu  display  in  a 
sequential  fashion  and  determine  the  overall  semantic  similarity,  x,  of  each  alternative  examined 
with  respect  to  the  target.  The  distribution  of  relatedness  values  among  targets  and  incorrect 
alternatives  for  a  given  menu  set  is  depicted  by  the  upper  curve  in  Figure  10.  The  lower  curve 
in  this  figure  represents  the  distribution  of  relatedness  values  between  targets  and  correct 
alternatives  for  the  same  menu  set.  Both  curves  in  this  figure  are  Gaussian. 


Weak  ^ —  Semantic  Relationship — ►  Strong 
High  ^ —  Semantic  Distance  — ►  Low 


C° 


Areas  under  the  distribution 


Weak  •< —  Semantic  Relationship — ►  Strong 
High  ^ —  Semantic  Distance  — ►  Low 

Legend. 

CO  -  Criterion  for  immediate  rejection.  All  alternatives  whose  relatedness  value  falls  to 
the  left  of  this  criterion  are  predicted  to  be  immediately  rejected. 

Cl  -  Criterion  for  immediate  selection.  All  alternatives  whose  relatedness  value  falls  to 
the  right  of  this  criterion  are  predicted  to  be  immediately  selected  (thus  leading  to  self-terminating 
search). 


A  through  F  -  Areas  defined  under  the  probability  distr.outions  by  the  two  criteria. 

Figure  10.  Areas  Defined  Under  the  Distribution  Curves  for 
the  Two-Criterion  Menu  Model. 

The  criterion  CO  in  Figure  10  represents  the  criterion  for  immediate  rejection.  Any  alternative 
whose  semantic  relatedness  to  the  target  (x)  falls  below  this  value  (x  <  CO)  will  be  immediately 
rejected  as  a  response  candidate  The  menu  search  for  the  correct  alternative  will  therefore 
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continue.  Cl  represents  the  criterion  for  immediate  selection  of  an  alternative  without  further 
evaluation  of  remaining  alternatives.  T'  "'',  evaluation  of  such  alternatives  is  predicted  to  result 
in  a  self-terminating  search  (x  >  Cl),  hvaluation  of  alternatives  with  relatedness  values  between 
these  criteria  results  in  the  continued  search  of  additional  alternatives.  If  none  of  the  remaining 
alternatives  searched  have  relatedness  values  that  exceed  C1,  the  alternative  possessing  the 
highest  degree  of  similarity  to  the  target  will  be  selected,  provided  its  relatedness  value  exceeds 
CO.  Exhaustive  searches  result  when  all  but  one  of  the  alternatives  are  rejected  without  further 
consideration  and  the  similarity  between  the  target  and  the  remaining  alternative  is  sufficient  to 
continue  search  but  insufficient  to  trigger  self-termination.  Exhaustive  searches  are  also  predicted 
for  situations  where  all  alternatives  are  rejected  (x  <  CO),  resulting  in  a  zero  response.  When 
none  of  the  alte.  natives  exceeds  the  rapid  selection  criterion  C1  and  similarity  values  for  more 
than  one  of  the  alternatives  fall  between  criteria,  all  of  these  latter  alternatives  are  re-evaluated, 
resulting  in  a  predicted  redundant  search. 

The  probability  that  a  correct  or  an  incorrect  alternative  falls  within  a  specific  range  of 
semantic  relatedness  values  is  identified  by  areas  A  through  F  in  Figure  10.  These  areas  are 
functions  of  the  parameters  which  characterize  the  two  Gaussian  distributions  depicted  in  this 
figure  and  the  location  of  selection  criteria.  Formulas  for  computing  these  areas  are  given  in 
Figure  11.  These  areas  can  be  used  to  estimate  the  probability  that  a  single  alternative  from 
a  given  menu  set  has  a  relatedness  value  that  is  below,  in  between,  or  above  criteria  vciues. 
In  addition,  when  values  for  Number  of  Alternatives  and  Omission  Probability  are  known,  probability 
functions  associated  with  areas  A  through  F  can  be  used  to  estimate  the  probabilities  of 
combinations  of  search  strategies  (self-terminating,  exhaustive  and  redundant)  and  response 
outcomes  (hit,  commission  miss,  false  alarm,  omission  miss,  and  ccmct  rejection)  for  a  given 
menu  set. 

Area  A  = 


Area  B  = 


Area  C  = 


Area  D  = 


CO 


Cl 

I  e  d:; 

J 

Cij 


'  -  )  c  :  -fX  l  1  = 

e  dx 

Cl 

CO 

r  C -.-yXcl^ 

1  e  dv; 


Cl 

P  -Cl)  C::-|.'Xc]  = 
urea  E  =  j  e  d;; 

CO 
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Legend. 


Area  E 


-<'i)  tx-pXcl’ 
e  d;-: 


CO  -  Rapid  rejection  criterion  value.  ci 
Cl  -  Rapid  selection  criterion  value. 

/<Xc  -  Mean  relatedness  value  for  correct  alternative  distribution. 
//Xi  -  Mean  relatedness  value  for  incorrect  alternative  distribution. 


Figure  1 1  Equations  for  Computing  Areas  Under  Probability 
Distributions  for  the  Two-Criterion  Menu  Model. 
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Eleven  combinations  of  search  strategy  and  response  outcomes  were  predicted  by  the  model. 
For  self-terminating  and  redundant  searches,  the  model  predicts  that  only  hits,  commission  misses 
and  false  alarms  occur.  For  exhaustive  searches,  predicted  response  outcomes  include  these 
outcomes  as  v/ell  's  omission  misses  and  correct  rejections.  Model  functions  were  derived  which 
predict  the  prob  lity  of  occurrence  for  these  search  and  response  outcome  combinations. 3 
The  resulting  r-  .ability  hmetiens  for  the  11  predicted  search  strategy  and  response  outcome 
combinations  ...le  given  t  equations  1,  2,  3,  4,  5,  6,  7,  8,  14,  15  and  18  listed  in  Table  16. 

The  predicted  effects  that  variations  in  the  model  distributions  have  on  discriminability  among 
targets  and  auernatives  are  straightforward.  In  Figure  10,  as  the  mean  semantic  distance  value 
for  the  correct  alternative  distribution  varies  from  low  to  high,  the  distribution  shifts  from  right 
to  left.  The  inverse  relationship  between  semantic  distance  values  and  semantic  similarity  should 
be  noted.  The  shift  in  the  correct  alternative  distribution  results  in  a  decrease  in  discriminability 
due  to  the  increased  overlap  of  this  distribution  with  the  distribution  representing  incorrect 
alternatives.  A  similar  effect  can  be  produced  as  the  mean  semantic  distance  value  for  the 
incorrect  alternative  distribution  varies  from  high  to  low.  The  model  assumes  that  subjects  adjust 
criteria  to  variations  in  menu  item  discriminability.  However,  the  model  does  not  assume  that 
changes  in  dis  'iminability  resulting  from  variations  in  either  distribution  have  an  identical  effect 
on  criteria,  or  tnat  changes  in  each  distribution  affect  the  criteria  in  a  different  manner.  Resolution 
of  this  issue  was  determined  empirically  using  procedures  described  later  in  this  section. 

Variations  in  the  Number  of  Alternatives  and  Omission  Probability  are  also  predicted  to  affect 
criteria  and  response  outcome  probabilities.  The  overall  trial  probabilities  of  an  incorrect  menu 
alternative  falling  within  a  specific  range  of  similarity  values  changes  as  a  function  of  the  Number 
of  Alternatives.  This  results  in  an  adjustment  of  criteria.  Variations  in  Omission  Probability  have 
a  similar  effect  on  the  overall  outcome  probabilities,  resulting  in  adjustments  to  selection  criteria. 

Although  selection  criteria  are  predicted  to  adjust  to  variations  in  Alternatives,  Omission  Prob¬ 
ability,  and  the  two  model  distributions,  there  is  no  theoretical  basis  for  predicting  the  specific 
nature  of  these  adjustn  ants.  The  effect  that  variations  in  these  parameters  have  on  model 
criteria  was  examined  empirically  using  data  from  the  menu  task  experiment  described  in  Section 
III.  Parameter  estimates  for  the  two  distributions  were  used  with  values  for  the  Number  of 
Alternatives  and  Omission  Probability  to  fit  performance  data  observed  for  individual  cells  of  the 
menu  task  experiment  to  model  predictions.  Best-fit  solutions  were  derived  by  varying  model 
criteria  and  other  model  parameters  to  minimize  the  differences  between  predicted  and  observed 
menu  task  performances.  Once  best-fit  solutions  were  derived,  relationships  between  model 
criteria  and  other  model  parameters  were  examined  through  a  series  of  regression  analyses. 


Data-Fitting  Methods 

Predicted  probabilities  of  menu  task  search  strategies  and  response  outcomes  for  variations 
in  Alternatives,  Omission  Probability,  and  areas  A  through  F  under  the  two  curves  in  Figure  10 
are  given  by  the  probability  functions  listed  in  Table  16.  Within  the  framework  of  these  equations 
there  are  a  number  of  ways  in  which  predictions  could  have  been  derived.  What  follows  is  a 
description  of  two  methods  that  were  used  in  the  present  study.  Both  are  derivations  of  fitting 
methods  reported  by  Smith  et  al.  (1974). 

Method  A 


Method  A  used  semantic  similarity  distances  derived  from  the  scaling  procedures  described 
in  Section  III  to  compute  mean  and  standard  deviation  estimates  for  the  two  model  distributions. 


detailed  summary  cf  the  procedures  used  to  derive  model  equations  appears  in  Pierce  (1989) 
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Table  16.  Probability  Functions  Search  Strategies  by 
Response  Outcomes  for  the  Two-Criterion  Menu  Model 


Number 

Description 

Function 

1 

P(ST  Hit) 

a-1 

[(l^a)  F  +  (  2  (l^a)  F(l-C)i)]  [1-Po] 
i=  1 

2 

P(ST  CM) 

a-2 

[C^  a]  [(a-  1)  +  (  2  (a-i-1)(1-C)i)  + 
i-  1 

a-2 

(1-F)(1  +  (  2  (i-H  1)(1-C)i))]  [l-Po] 
i=  1 

3 

P(ST  FA) 

a-1 

[C]  [1  +  (  2  (1-0*)]  [Po] 
i=  1 

4 

P(Ex  Hit) 

(E(A^-i)]  (1-Poj 

5 

P(Ex  CM) 

((a-1)DB(Aa-2)]  [1-Poj 

6 

P(Ex  FA) 

[aB(A®-i)|  [Po] 

7 

P(Ex  OM) 

[D(A«-’)]  [1-Po] 

8 

P(Ex  CR) 

[A«]  [Po] 

9 

P(Red  Search, 

CA  Present  and 

C0<  Xc<  Cl) 

a-1  a-1 

[E  2  (  )  *  B*]  [1-Po] 

i=  1  i 

10 

P(Red  Hit  |  a  =  2 
and  CA  present) 

Cl  X 

f  r  f  -(i/2)[y-//Xi]2  1  -(i/2)[x-/<Xc]2 

dy 


dx 


CO 

CO 

11 

P(Red  CM  1  a  =  2 

Cl 

Cl 

and  CA  present) 


r 

-(i/2)[y-/rXi]2 

J  L 

e 

dy 

CO 

12 

P(CO<  m  Xi 

Cl 

values<  Xc<  Cl  ( 

c 

CA  present) 

-(i/2){y-/rXi]2 


-(1/2)[X-/<Xc)2 
e  dx 


dy 


m  -(i/2)[x-//Xc]^ 
e 


dx 


CO  CO 
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Table  16.  (Concluded) 


Number  Description 


13  P(C0 <  Xc<  and 

C0<  m  Xi  values < 
Cl  and  at  least 
1  Xi<  Xc 
|CA  present) 


Function 


Cl  Cl 


m  m 
2  {  ) 
i  =  1  i 


Co  X 


-(i/2)[y-/<Xi] 


-(l/2)[Z-/vXi]' 


m-i  -(i/2)[x-//Xc]^ 


P(Red  Hit) 


(  )  (EQ  12;  m=  i)]  [1-Po] 


P(Red  CM  |a=  2) 


[EQ  13;  m=  1]  [1-Po] 


P(Red  CM  |a>  2) 


((EQ  16)  +  (EQ  17))  [1-Po] 


P(Red  CM  and 
C0<  Xc<  Cl  I 
CA  present) 


a-1  a-1 

[  2  (  )  A«-'-’  (EQ  13:  m=  i)J 

i  =  1  i 


P(Red  CM  and 
Xc<  CO  I 
CA  present) 


a-1  a-1 

[  2  (  )  A®-'-'' 

i=  2  i 


P(Red  FA) 


Abbreviations 

In  Descriptions. 

ST  -  Self-terminating  search 

Ex  -  Exhaustive  search 

Red  -  Redundant  search 

CM  •  Commission  miss 

FA  -  False  alarm 

OM  -  Omission  miss 

CR  -  Correct  rejection 

CA  -  Correct  alternative 

lA  -  Incorrect  alternative 

Xc  -  Semantic  relatedness  value  of  CA 

Xi  -  Semantic  relatedness  value  of  lA 

CO  •  Criterion  for  immediate  rejection 

C1  Criterion  for  rapid  selection 

Srch  -  Search 

P  -  Probability 


P(Red  FA)  = 


a-2  a 

[  2  (  )  B^  '  A']  [Po] 

i=  0  i 


In  Functions 

A,  B,  C,  D,  E,  F  -  Areas  defined  by 
distributions  and  criteria 
a  -  Number  of  alternatives 
Po  •  Probability  of  correct  alternative  absent 
/rXc  -  Mean  relatedness  value  for  CA  distribution 
//Xi  -  Mean  relatedness  value  for  lA  distribution 
EQ  -  Equation 


Distribution  estimates  were  computed  separately  for  menu  stimuli  used  in  each  cell  of  the  menu 
task  experimental  design.  The  distribution  representing  the  relationship  between  targets  and 
correct  alternatives  was  estimated  for  each  cell  by  computing  the  mean  and  standard  deviation 
of  Dc  values  (scaled  distance  from  target  to  correct  alternative)  associated  with  trials  comprising 
that  cell.  Similarly,  the  distribution  representing  the  relationship  between  targets  and  incorrect 
alternatives  was  estimated  for  each  cell  by  computing  the  mean  and  standard  deviation  of  the 
Di  values  (scaled  distances  from  correct  to  each  incorrect  alternative)  associated  with  each 
incorrect  alternative  across  trials  in  each  cell. 

Under  f^.v^thod  A,  distribution  means  and  standard  deviations,  along  with  Alternatives  and 
Omission  Probability  values  for  each  cell,  were  fixed  during  computation  of  best-fit  solutions. 
Values  for  the  two  criteria  CO  and  C1  were  free  to  vary  until  differences  between  predicted  and 
observed  performance  proportions  were  minimized. 

Combined  search  strategy  by  response  outcome  proportions  corresponding  to  the  combinations 
predicted  by  the  model  equations  listed  in  Table  16  were  computed  for  individual  subjects  for 
individual  cells  using  only  the  sequential  Presentation  Mode  data  from  the  menu  task  experiment. 
These  proportions  were  operationally  determined  using  the  identical  definitions  of  search  strategy 
and  response  outcome  described  in  Section  IV.  It  should  be  remembered  that  the  case  where 
a  subject  examined  all  alternatives  once  and  only  once  and  chose  the  alternative  in  the  last 
position  in  the  menu  was  considered  to  be  an  indeterminate  search.  However,  discarding  these 
data,  as  was  done  with  analyses  of  menu  task  search  performances  (see  Section  IV),  would 
artificially  increase  cell  proportions  of  redundant  searches  associated  with  the  data  set.  Additionally, 
discarding  these  data  would  have  affected  the  accuracy  of  computed  response  outcome  proportions. 
Therefore,  the  decision  was  made  to  adjust  observed  self-terminating  and  exhaustive  search  data 
to  account  for  the  number  of  indeterminate  search  trials  observed.'* 

Mean  proportions  for  each  of  the  1 1  possible  combinations  of  search  strategy  and  response 
outcome  were  computed  across  subjects  for  individual  cells  and  used  as  input  to  the  left  side 
of  the  model  equations  presented  in  Table  16  to  derive  estimates  of  the  two  criteria  CO  and 
Cl.  Best-fit  solutions  were  derived  by  initially  setting  CO  to  the  Di  distribution  mean  and  Cl  to 
the  Dc  distribution  mean,  and  then  systematically  varying  criteria  values  until  a  best-fit  criterion 
was  met.  The  fitting  procedure  used  an  ordinary  least-squares  method  where  the  sum  of  the 
square  error  values  computed  between  predicted  and  obtained  proportions  across  all  prediction 
equations  was  minimized  for  each  trial  cell.  Mean  proportions  from  a  total  of  32  cells  (representing 
all  combinations  of  Alternatives,  Omission,  Hierarchy,  and  Nesting  factor  levels  in  the  menu  task 
experiment)  were  fitted  for  sequential  Mode  data. 


“^Because  search  strategy  for  indeterminate  search  trials  could  be  considered  either  self-terminating  or 
exhaustive,  proportions  of  response  outcomes  for  self-terminating  and  exhaustive  searches  were  adjusted  for 
individual  subjects  within  each  cell  based  on  the  number  of  indeterminate  search  strategy  trials  observed  for 
each  type  of  response  outcome.  To  the  extent  that,  by  definition,  indeterminate  search  strategies  contained 
only  responses  where  an  alternative  was  chosen  (i.e.,  hits,  commission  misses,  or  omission  misses),  only  data 
associated  with  these  types  of  responses  were  affected  by  this  procedure.  In  the  sequential  Presentation  Mode 
database,  1.318  of  a  total  of  6.144  trials  resulted  in  indeterminate  search  strategy. 

For  each  subject,  a  ratio  was  computed  for  each  cell  of  observed  self-terminating  search  trials  divided  by 
the  sum  of  both  self-terminating  and  exhaustive  search  trials  on  which  an  alternative  was  chosen.  The  count 
of  self-terminating  search  trials  for  each  of  the  three  response  outcomes  affected  was  then  adjusted.  This  was 
done  by  adding,  to  the  determinable  number  of  self-terminating  search  trials  observed  for  each  of  these  response 
outcomes  the  number  of  indeterminate  search  strategy  trials  observed  for  that  response  outcome  type  multiplied 
by  this  ratio.  Similarly,  for  each  subject,  a  ratio  was  computed  of  observed  exhaustive  search  trials  divided 
by  the  sum  of  both  exhaustive  plus  self-terminating  search  trials  on  which  an  alternative  was  chosen.  The 
count  of  exhaustive  search  trials  for  each  of  the  three  response  outcomes  was  then  adjusted  by  adding,  to  the 
determinable  number  of  exhaustive  search  trials,  the  number  of  indeterminate  search  strategy  trials  observed  for 
that  response  outcome  multiplied  by  the  ratio.  Proportions  for  all  search  strategy  and  response  outcomes  were 
then  computed  by  dividing  the  adjusted  counts  by  the  total  number  of  trials  performed  for  the  cell. 
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Method  B 


Model-fitting  procedures  conducted  under  Mettiod  B  were  identical  to  those  for  Method  A 
in  all  respects  except  the  number  of  free  parameters  used  to  derive  best  fit  solutions.  In  addition 
to  Co  and  Cl,  the  standard  (ieviation  parameters  representing  the  Dc  and  Di  distributions  were 
also  included  as  free  parameters  in  Method  B.  Best-fit  solutions  were  derived  by  initially  setting 
values  for  these  standard  deviations  to  the  values  used  under  Method  A,  and  then  systematically 
varying  these  values  along  witfi  CO  and  Cl  values  until  a  best-fit  criterion  was  met.  The  intent 
of  this  method  was  to  determine  the  extent  to  wiiicii  model-fitting  improved  as  standard  deviation 
parameters  were  allowed  to  vary.  Improvement  was  assessed  by  comparing  the  root  mean 
square  of  the  differences  between  predicted  and  obtained  performances  across  cells  for  each 
solution 


Procedures 

A  computer  program  v;as  developed  to  compute  separate  fitting  solutions  for  data  obtained 
from  each  cell  of  the  experimental  design  using  Methods  A  and  B.  The  program  derived 
solutions  for  integrals  associated  with  the  equations  in  Figure  11  and  Table  15  through  numeric 
approximation  ^  Best-fit  values  for  free  parameters  were  found  by  a  hill-climb  method.  In  the 
hili-clirnb,  free  parameters  were  sequentially  incremented  by  a  delta.  After  each  increment,  the 
fit  was  evaluated  by  summing  the  squared  deviations  between  predicted  and  observed  proportions. 
If  the  fit  improved,  the  parameter  was  set  to  the  new  value.  If  the  fit  did  not  improve,  the 
original  value  of  the  parameter  was  decremented  by  the  delta  and  the  fit  was  recomputed.  If 
this  resulted  in  a  fit  improvement,  the  parameter  was  set  to  the  decremented  value.  If  the  fit 
did  not  improve,  the  parameter  was  set  to  its  original  value  This  was  repeated  for  each  free 
parameter,  in  turn,  until  a  pass  was  made  that  resulted  in  no  fit  improvement  At  this  point,  the 
delta  was  decreased  and  tfie  process  was  repeated  until  the  delta  reached  a  preset  minimum, 

Tfie  program  computed  fitting  solutions  using  Methods  A  and  B  for  several  subsets  of  the 
database  in  order  to  address  certain  problems  associated  with  the  design  of  the  menu  task 
experiments^  it  may  be  recalled  that  analysis  of  Omission  Probability  manipulation  effects  on 

^'Spc,,fuse  of  the  inverse  relationship  between  semantic  distance  and  semantic  similarity,  the  computation  of 
ni, meric  approximations  fo:  integrals  associated  with  equations  in  Figure  11  and  Table  16  required  that  all 
distribution  means  that  were  estimated  for  each  of  the  32  cells  be  transformed  to  negative  values  (i.e.,  multiplied 
by  -1'  Thus,  an  estimated  dislnbufion  mean  of  1  as  tfaosformert  to  15  This  had  the  effect  of  reversing 

the  original  relatioriship  between  distance  values  ai  semantic  relationships.  For  e.xample,  while  a  change  in 
the  l;c  me.an  from  1  5  to  1  prio-  to  the  translormation  reflected  a  decrease  in  semantic  distance,  it  also  reflected 
.an  ncre  ue  in  semantic  a.miian  /.  Attei  transformation  of  these  values,  the  change  from  -15  to  -1  resulted  in 
botti  an  I'-.iease  in  ttie  tninst-  -me,!  value  is  vie''  as  an  increase  m  semantic  similarity  This  transformation 
.nrocedu  e  resulted  m  denv.at.on  ct  ci  te'  .a  “stiniates  whose  values  also  related  directly  to  levels  of  semantic 
sim.iarit/  upon  comp'etion  o'  co oip'jt.itions  to  derive  best-fit  solutions,  the  same  transformation  was  conducted 
1  second  t  rue  on  ..rlues  re  '.tiSiiiPulK.-  means  and  model  criteria  Tfuis,  all  values  tor  distribution  means  and 
nc.de!  cnteri.a  reporterj  ir,  the  reflect  values  that  are  inversely  related  to  semantic  similarity;  i.e..  increases 

ri  vaiues  for  distribut  on  f'lpar.s  .nr  cnien.j  retiect  decreases  in  semantic  similarity 

^In  addition  to  those  fitting  soi..jtions  reported  in  the  main  body  of  this  text.  Method  A  and  B  fitting 
.lolutior'is  aiso  were  computed  across  aii  tr  als  foi  e.ich  ceil  using  estimates  ot  distribution  parameters  for  Correct 
Alternative  Present  triais  orl ,  In  'hese  solutions  data  in  all  but  t1  cells  were  fitted  to  equations  1,  2,  3.  4, 
5  6  '  8.  14  15.  and  18  n  r-ir  ,p  m  i;,  cc'is  whe'e  Unnssiori  Prohahiiity  erjualed  zeio.  and  in  cells  where 
Number  f  *  Alternatives  ech.  ued  o'or-t  nnd  f  "  .-r  e- Kcr'y  f.actor  le-.ei  w.as  riu'-o  ,a')  tri.als  had  menus  where  the 
Cg  :*jc  Aitfi  native  ww  Pe-e.ii  v  ,.  i:  cei's  (fata  were  ,’,tied  tirniy  to  the  7  tnndel  equations  that 

predicted  prob.aoi’ities  as.r.pm  I'ed  w^tn  tr,a'S  -.-diefe  p.p  Jorrec!  Aitern.ative  vms  Frese",t  thus  probability  functions 
t  ■(  t.aise  .jiarrii  and  ij'ir'e  n  rp.prtion  oulconies  vvere  .srn'tted  fr-rni  tt'e- e  >-ol‘.iro.’is  -.r'd  r'riiy  equations  1.  2,  4. 
5.  7  14  arid  15  ir'  lab  Ih  were  ’i-;erj 

Ip  .iddition  sepa-ate  Methrj'i  rS  a'nl  H  srh.itr-ons  were  rrmipiited  t  n  ‘torroct  .Alternative  Absent  data  using 
distribution  parametei  estimates  ‘c  C  otre.pt  Alternative  .Absent  trials  'isinq  equ-ations  1.  6,  8.  and  18  in  Table 
In  Because  o*  I'  e  limitort  '.umi  er  o*  Ph-ervafions  per  r  pij  for  C'-rtpr^  Alternative  Absent  trials  (ranging  from 
zero  to  twelve)  estrPuatr  :,  o(  “  e  it.  d  ■.•nt.vt:  ■■  .p.-arameiprs  used  m  co.mprjting  ttiese  s'liutions  were  questionable. 

Resuitmg  fits  t.  '  the  ad  ti -r-  s.iirrhons  were  .highly  similar  to  those  ottriined  for  the  Correct  Alternative 
present  tii.ai  solutions  reported  in  the  text  However  solutions  for  the  Correct  Alternative  Absent  trial  data  were 
relatively  poor  and  mt  remliiy  in»erorp!ahle  Due  to  the  problems  associated  with  both  of  these  sets  of  solutions, 
ctisrijcsion  m  'he  text  was  I"  oniv  .Meifind  ■  and  B  solutions  where  proporidcps  t  i  only  Correct  Alternative 

Preser.f  trials  were  'Cted 
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Dc  and  Di  values  in  the  menu  task  experiment  showed  significant  differences  in  mean  Di  values 
between  Correct  Alternative  Present  and  Correct  Alternative  Absent  trials.  These  results  suggest 
that  the  Di  distributions  for  Correct  Alternative  Present  and  Absent  trials  were  not  the  same. 
Though  separate  estimates  of  Di  distribution  parameters  for  the  Correct  Alternative  Absent  trials 
were  available,  the  limited  number  of  trials  per  cell  (ranging  from  0  to  12)  that  were  available 
to  estimate  these  parameters  made  the  reliability  of  these  estimates  questionable.  However,  a 
relatively  clean  data  set  for  which  large  numbers  of  trials  per  cell  were  available  was  obtained 
using  data  from  Correct  Alternative  Present  trials.  Therefore,  discussion  will  focus  only  on 
Method  A  and  B  solutions  derived  for  all  cells  using  data  from  trials  only  where  the  Correct 
Alternative  was  Present.  For  these  solutions,  probability  functions  for  false  alarms  and  correct 
rejection  outcomes  were  omitted  and  only  equations  1,  2,  4,  5,  7,  14,  and  15  in  Table  16  were 
used. 

Another  problem  that  needed  to  be  addressed  concerned  the  fact  that  the  model  predicted 
zero  choice  responses  (none  of  the  alternatives  are  chosen  as  correct)  only  under  conditions 
where  an  exhaustive  search  was  made.  Whereas  over  90%  of  zero  choice  responses  were 
made  after  an  exhaustive  search  of  the  alternatives,  106  out  of  a  total  of  1,106  zero  choice 
responses  were  made  after  a  redundant  search.  These  responses  were  not  equally  distributed 
across  cells.  Thus,  in  order  to  derive  best-fit  solutions  with  comparable  measures  of  fit  across 
all  cells,  redundant  search  trials  where  zero  choice  responses  were  made  were  deleted  from 
the  database.  The  remaining  data  sets  were  then  normalized  such  that  the  proportions  for  the 
search-strategy-by-response-outcome  categories  summed  to  one. 

In  summary,  proportions  of  search  strategies  and  response  outcomes  for  Correct  Alternative 
Present  trials  observed  for  each  of  the  32  design  cells  were  used  to  derive  criteria  estimates 
using  Method  A  and  B  fitting  techniques.  Thus,  Correct  Alternative  Present  data  were  fitted  to 
equations  1,  2,  4,  5,  7,  14,  and  15.  These  equations  correspond  only  to  search-strategy- 
by-response-outcome  combinations  predicted  when  the  Correct  Alternative  was  Present  in  the 
menu.  Distribution  parameters  were  derived  from  the  scaled  estimates  of  menu  stimuli  semantic 
distances  for  Correct  Alternative  Present  trials  in  each  cell.  Finally,  all  observed  proportions  for 
each  cell  were  normalized  to  sum  to  one. 


Findings 


Model  Solutions 


Root  mean  squared  error  (RMSE)  values  and  correlations  between  predicted  and  obtained 
search-strategy-by-response-outcome  proportions  were  used  as  indicants  of  the  fit  between 
empirical  data  and  model  predictions.  RMSE  values  represent  the  root  mean  of  the  squared 
differences  between  predicted  and  observed  proportions.  Tables  listing  model  parameter  and 
RMSE  values  that  were  derived  from  Method  A  and  B  solutions  for  individual  cells  are  given  in 
the  Appendix.  In  the  tables  in  the  Appendix,  criteria  values  represent  semantic  distance  measures 
and,  therefore,  have  the  same  inverse  relationship  to  semantic  similarity  as  do  Dc  and  Di  values. 
As  values  for  either  criterion  increase,  the  degree  of  semantic  similarity  required  to  exceed  the 
criterion  decreases.  Also  included  in  the  Appendix  are  RMSE  and  correlation  values  computed 
between  observed  and  predicted  proportions  for  individual  equations  across  cells. 

RMSE  values  computed  across  all  seven  equations  and  all  cells  were  .079  and  .018  for 
Method  A  and  B  solutions,  respectively.  Correlations  between  predicted  and  observed  proportions 
across  all  seven  equations  and  all  cells  were  .887  for  the  Method  A  solutions  and  .994  for  the 
Method  B  solutions.  Though  the  data  used  to  compute  these  correlations  were  not  independent 
(within  each  cell,  the  sum  of  the  proportions  for  all  equations  equaled  one),  the  correlations 
are  reported  here  to  serve  as  a  measure  of  congruency  between  predicted  and  observed 
proportions. 
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Effects  of  Menu  Design  Factors  on  Model  Criteria 


The  effects  that  variations  in  the  Number  of  Alternatives,  Omission  Probability,  and  the  means 
and  standard  deviations  of  the  two  model  distributions  had  on  model  criteria  were  examined 
through  a  series  of  regression  analyses.  Data  used  for  these  regressions  were  derived  from  the 
best-fit  model  solutions  of  menu  task  performance  data.  A  total  of  four  regressions  were 
computed  using  data  derived  for  individual  cells  as  the  basic  unit  of  analysis.  In  two  of  the 
regressions,  the  value  of  the  rapid  rejection  criterion,  CO,  that  was  derived  for  individual  cells 
was  used  as  the  criterion  measure.  In  the  other  two  regressions,  the  criterion  measure  used 
was  the  value  of  the  rapid  selection  criterion.  Cl,  derived  for  individual  cells.  For  each  of  these 
two  criterion  measures,  one  regression  was  computed  using  parameter  estimates  from  the  Method 
A  solution  and  another  regression  was  computed  using  parameter  estimates  from  the  Method 
B  solution.  In  all  four  regressions,  predictor  variables  were  Number  of  Alternatives,  Omission 
Probability,  the  Dc  mean,  the  Dc  standard  deviation,  the  Di  mean,  and  the  Di  standard  deviation 
computed  for  each  cell. 

Results  of  regressions  for  the  rapid  rejection  criterion,  CO,  are  summarized  in  Table  17  for 
both  Method  A  and  B  solutions.  Significant  F-values  were  obtained  for  both  regressions.  R-squares 
for  these  regressions  indicated  that  over  95%  of  the  CO  variance  was  accounted  for  by  model 
predictors. 

Comparison  of  R-square  values  between  the  two  CO  regressions  indicated  a  high  degree  of 
consistency.  In  addition,  signs  of  the  significant  partial  coefficient  estimates  were  consistent 
across  the  two  regressions.  These  findings  notwithstanding,  the  significance  of  the  partial 
correlations  and  coefficients  for  individual  predictors  differed  somewhat  across  the  two  regressions. 
In  regressions  computed  for  the  Method  A  solution,  all  predictors  with  the  exception  of  the  Di 
standard  deviation  significantly  contributed  to  the  regression  model,  and  all  but  the  Number  of 
Alternatives  predictor  significantly  contributed  to  the  regression  model  for  the  Method  B  solution. 

Results  of  regressions  for  the  rapid  selection  criterion.  Cl,  are  summarized  in  Table  18  for 
the  Method  A  and  B  solutions.  Significant  F-values  were  obtained  for  both  regression  models. 
R-squares  were  .49  for  the  Method  A  data  and  .74  for  the  Method  B  data. 

There  were  substantial  differences  obtained  between  the  Cl  regression  results  for  Method  A 
and  B  solution  data.  Regression  models  computed  for  Method  B  data  accounted  for  over  24% 
more  variance  than  did  regressions  for  Method  A  data.  Across  both  regressions,  the  partial 
correlations  and  coefficients  for  the  Number  of  Alternatives  predictor  were  significant.  However, 
whereas  regression  results  for  the  Method  A  solution  also  indicated  that  the  partial  correlations 
and  coefficients  for  the  Di  standard  deviation  were  significant,  these  statistics  were  not  significant 
for  this  predictor  in  the  regression  results  for  Method  B  solutions.  Finally,  regressions  computed 
for  Method  B  solution  data  indicated  that  the  partial  correlations  and  coefficients  for  the  Dc 
mean  were  significant.  In  contrast,  these  statistics  were  not  significant  in  regressions  of  Method 
A  solution  data. 


Synopsis 

In  the  absence  of  a  well-defined  criterion  measure  or  competing  model  whose  predictions 
could  be  used  for  comparison  to  the  results  obtained  for  the  two-criterion  menu  model,  it  is 
difficult  to  evaluate  goodness  of  fit  between  predicted  and  observed  task  performances.  Even 
so,  RMSE  and  correlation  values  between  model  predictions  and  observed  proportion  data  did 
provide  a  means  of  comparing  differences  in  fits  derived  for  the  two  fitting  methods  used. 

Differences  between  regression  results  for  the  two  fitting  methods  were  obtained  for  both 
criterion  measures.  Whereas  in  Method  A  only  model  criteria  were  free  to  derive  best-fit  solutions. 
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Table  18.  Regression  Results  for  Rapid  Selection  Criterion  (Cl)  Estimates  from  Method  A  and  B  Solutions 
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in  Method  B  both  the  model  criteria  and  the  standard  deviation  parameters  were  free  to  derive 
fitting  solutions.  Though  it  is  tempting  to  attribute  the  better  fits  obtained  for  Method  B  solutions 
solely  to  the  optimization  of  standard  deviation  estimates  for  the  two  model  distributions,  such 
a  conclusion  would  be  difficult  to  support.  Similar  improvement  in  fits  could  have  been  demonstrated 
by  fixing  the  standard  deviation  parameters  and  freeing  up  parameters  representing  the  means 
of  the  two  model  distributions.  Thus,  improvements  in  the  Method  B  fits  could  be  attributed  not 
only  to  better  estimation  of  the  standard  deviation  parameters,  but  also  to  improvements  that 
could  have  been  derived  from  better  estimation  of  distribution  means.  Finally,  it  could  be  argued 
that  the  observed  improvements  in  fit  may  have  been  due  simply  to  a  better  accounting  of  error 
between  predicted  and  obtained  proportions  that  cannot  be  attributable  to  improvement  in  the 
accuracy  of  estimating  any  of  the  model  parameters.  Although  these  results  suggest  the 
possibility  that  there  may  exist  better  measures  that  could  be  used  to  represent  model  distribution 
parameters,  the  difficulty  in  identifying  the  real  source  of  these  fit  improvements  leaves  resolution 
of  this  issue  to  further  research. 

The  issue  raised  above  has  further  implications  concerning  interpretation  of  the  results  of 
regressions  computed  for  the  two  model  criteria.  Because  the  standard  deviation  estimates 
derived  from  Method  B  solutions  cannot  be  interpreted  conclusively  as  reflecting  values  that 
represent  the  actual  standard  deviations  of  these  distributions,  the  relationship  between  regression 
model  predictions  and  the  two  criteria  is  clearly  better  reflected  by  regressions  performed  on 
the  data  derived  from  Method  A. 

Focusing  on  the  partial  coefficients  tested  in  this  latter  set  of  regressions,  the  rapid  rejection 
criterion.  CO,  was  shown  to  be  significantly  affected  by  variations  in  all  predictors  except  the 
one  representing  the  Di  standard  deviation.  CO  increased  (reflecting  a  decrease  in  the  degree 
of  semantic  similarity  required  to  exceed  it)  as  the  Number  of  Alternatives  and  Omission  Probability 
decreased.  CO  also  increased  as  the  Dc  standard  deviation  and  both  distribution  means  increased. 
The  rapid  rejection  criterion.  Cl,  was  significantly  affected  only  by  variations  in  the  Number  of 
Alternatives  and  the  Di  standard  deviation.  As  Number  of  Alternatives  increased.  Cl  increased. 
As  the  Di  standard  deviation  increased,  C1  decreased.  It  appears  from  these  results  that  while 
all  factors  contributed  to  variations  in  menu  criteria,  their  individual  effects  were  more  or  less 
pronounced  depending  on  the  criterion  in  question. 

These  differences  are  further  exemplified  through  comparison  of  the  effects  that  distribution 
parameters  had  on  the  two  criteria.  Results  from  the  CO  regression  using  Method  A  data  suggest 
that  variations  in  the  Di  and  Dc  distributions  had  similar  effects  on  the  CO  criterion.  Thus,  it 
would  appear  that  overall  discriminability  among  alternatives  had  a  significant  effect  on  the  value 
of  CO.  Results  from  the  Cl  regression  indicate  that  whereas  variations  in  the  Di  standard  deviation 
significantly  affected  values  of  the  Cl  criterion,  Dc  distribution  parameters  did  not.  In  contrast 
to  the  results  obtained  for  CO  regression,  these  results  suggest  that  Cl  was  not  affected  as 
much  by  the  overall  discriminability  among  alternatives  (represented  by  variations  in  either  of 
the  model  distributions)  as  by  variations  in  the  Di  distribution  alone. 

In  summary,  the  fitting  procedures  and  regression  analyses  provided  a  means  of  assessing 
the  extent  to  which  data  could  be  fitted  to  model  equations,  as  well  as  a  means  of  assessing 
the  relationship  between  menu  design  factors  and  model  criteria  Comparison  of  the  results  of 
the  two  methods  used  to  compute  fitting  solutions  suggested  further  research  as  to  how  model 
distributions  might  best  be  estimated.  Finally,  examination  of  reg  .ion  results  obtained  for  the 
Method  A  solution  indicated  that  variations  in  model  criteria  are  significantly  related  to  variations 
in  Number  of  Alternatives,  Omission  Probability,  and  the  estimated  means  and  standard  deviations 
of  model  distributions. 


VI.  DISCUSSION 

The  previous  section  described  the  two-criterion  menu  model  and  procedures  by  which 
empirical  data  were  fitted  to  the  model  to  derive  estimates  of  model  criteria  for  cells  of  the 
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experimental  design.  Regressions  on  criteria  values  derived  from  these  procedures  showed  that 
significant  relationships  exist  between  model  criteria  and  menu  design  factors.  The  present 
section  uses  these  results  to  explicate  the  results  of  the  menu  task  performance  analyses 
described  in  Section  IV,  and  then  proposes  an  agenda  for  addressing  relevant,  unresolved  issues 
through  future  research. 


Explication  of  Menu  Task  Performance  Results 

As  described  in  the  previous  section,  model  criteria  values  were  derived  only  for  menu  task 
data  that  were  obtained  for  sequential  Presentation  condition  trials  in  which  the  Correct  Alternative 
was  Present.  Discussion  will  be  limited,  therefore,  to  performance  results  associated  with  these 
data.  These  results  pertain  to  proportions  of  hits,  self-terminating  searches,  exhaustive  searches, 
and  redundant  searches  for  trials  performed  under  the  sequential  Mode  condition  where  the 
Correct  Alternative  was  Present.  In  addition,  because  model  probability  functions  were  based  on 
semantic  distance  distributions  estimated  for  all  alternatives  in  each  menu,  only  performance 
results  for  models  containing  the  Di  average  predictor  are  considered.  The  effects  that  each 
of  the  four  menu  design  factors  (Number  of  Alternatives.  Omission  Probability,  semantic  distance 
between  target  and  correct  alternative,  and  average  semantic  distance  between  correct  and 
incorrect  alternatives)  had  on  model  criteria  and  model  predictions  are  each  considered  separately. 
The  effect  that  each  factor  had  on  menu  task  performance  is  then  explained  in  terms  of  the 
processes  suggested  by  the  model. 

The  relationships  between  model  criteria  and  areas  under  the  model  distribution  curves  were 
presented  in  Figure  10.  To  aid  the  discussion  of  these  relationships,  Figure  12  provides  a 
summary  of  results  from  menu  task  performance  and  model  criteria  regressions.  The  partial 
coefficients  for  the  four  menu  design  factors  that  were  derived  from  regressions  of  sequential 
mode  hit  and  search  strategy  proportions  (originally  presented  in  Tables  10  and  15)  are  presented 
in  Figure  12a.  Figure  12b  lists  the  partial  coefficients  computed  from  the  regressions  on  model 
criteria  values  that  were  derived  from  the  Method  A  fitting  procedure  (originally  presented  in 
Tables  17  and  18). 


Effects  of  Variations  in  Number  of  Alternatives 


Model  parameter  relationships.  The  curves  shown  in  Figure  10  represent  the  distributions 
of  relatedness  values  among  targets  and  correct  alternatives,  and  among  targets  and  incorrect 
alternatives.  As  the  number  of  incorrect  alternatives  in  the  menu  increases,  the  greater  the 
likelihood  that  at  least  one  of  the  incorrect  alternatives  in  the  menu  will  fall  within  any  one  of 
the  three  areas  defined  under  the  upper  curve  in  this  figure.  Thus,  as  the  Number  of  Alternatives 
increases,  the  greater  the  probability  that  at  least  one  incorrect  alternative  will  exceed  both  the 
value  of  the  correct  alternative  and  the  values  of  model  criteria. 

Additionally,  as  the  Number  of  Alternatives  increases,  subjects  require  increasing  relatedness 
between  target  and  alternatives  to  satisfy  requirements  of  the  CO  criterion.  This  finding  is 
reflected  in  Figure  12b  by  decreases  in  the  value  of  CO  as  Number  of  Alternatives  increases. 
Because  the  values  associated  with  both  criteria  were  set  to  the  same  scale  that  was  used  for 
values  of  Dc  and  Di,  decreases  in  criteria  values  reflect  increases  in  semantic  relatedness 
requirements.  Conversely,  the  relatedness  requirements  to  exceed  the  rapid  selection  criterion 
decrease  as  the  Number  of  Alternatives  factor  increases  (reflected  in  Figure  12b  by  increasing 
C1  values). 

Effects  on  Response  Accuracy.  As  the  Number  of  Alternatives  increases,  the  model  predicts 
fewer  responses  that  result  in  a  hit,  due  to  (a)  an  increase  in  the  likelihood  that  at  least  one 
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Response  Accuracy:  Hits  -  Sequential  mode  only 

PREDICTOR  VARIABLE^  Coef  Std  Err  t~ 

Alternatives  -.062  .012  -4.938 

Omission  Prob  -.908  .195  -4.647 
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incorrect  alternative  will  exceed  the  relatedness  value  of  the  correct  alternative  in  the  menu  (thus 
resulting  in  a  commission  miss)  and  (b)  an  increase  in  the  likelihood  of  an  omission  miss  as 
a  result  of  increases  in  semantic  relatodness  requirements  associated  with  the  CO  criterion.  With 
both  commission  misses  and  omission  misses  predicted  to  increase  with  increasing  Numbers  of 
Alternatives,  proportions  of  a  hit  must  decrease.  The  regression  results  for  proportions  of  hits 
are  consistent  with  model  predictions.  As  indicated  in  the  summary  of  response  accuracy 
regression  results  listed  in  Figure  12a,  proportions  of  hits  decreased  as  Number  of  Alternatives 
increased. 

Effects  on  Search  Strategies.  The  effects  of  increases  in  Number  of  Alternatives  on  search 
strategies  employed  by  subjects  during  menu  task  performance  are  a'so  predicted  by  the  model. 
The  regression  results  for  the  Cl  criterion  indicate  that  relatedness  'egU'rements  to  meet  this 
threshold  decreased  as  Number  of  Alternatives  increased.  Thus,  the  model  predicts  that  increases 
in  Numbers  of  Alternatives  results  in  lower  semantic  similarity  requirements  to  produce  a 
self-terminating  search.  In  addition,  with  more  incorrect  alternatives  in  the  menu,  the  greater 
the  probability  that  at  least  one  of  them  will  meet  the  rapid  selection  threshold.  The  effect  of 
increased  Number  of  Alternatives  obtained  from  analysis  of  self-terminating  search  performance 
data  are  consistent  with  these  predictions.  As  indicated  in  the  summary  of  search  strategy 
regression  results  listed  in  Figure  12a,  as  Number  of  Alternatives  increased,  self-terminating 
searches  significantly  increased. 

The  results  obtained  from  analysis  of  exhaustive  and  redundant  search  performance  data  for 
variations  in  the  Number  of  Aiiernatives  are  also  consistent  with  model  predictions.  The  regression 
of  model  parameter  data  on  the  CO  criterion  indicates  that  the  semantic  relatedness  requirements 
to  meet  this  threshold  increased  with  increasing  Numbers  of  Alternatives.  Furthermore,  as 
Number  of  Alternatives  increased,  the  relatedness  requirements  to  meet  the  Cl  rapid  selection 
criterion  decreased.  These  results  indicate  a  decrease  in  the  likelihood  that  any  one  particular 
alternative  will  have  a  reiatedness  value  that  (alls  between  criteria.  To  some  extent,  however, 
this  tendency  is  offset  by  the  predicted  increase  in  the  likelihood  that  with  more  Alternatives, 
the  probability  of  having  more  than  one  alternative  fall  within  any  given  range  of  relatedness 
values  increases.  Thus,  a  tradeoff  between  increased  Numbers  cf  Alternatives  competing  for 
selection  versus  a  decrease  in  the  range  of  values  that  define  the  area  under  the  distributions 
between  criteria  is  suggested  by  the  model. 

Results  of  the  analysis  of  exhaustive  and  redundant  search  data  indicate  that  increases  in 
the  Number  of  Alternatives  resuited  in  an  increase  in  the  overall  probability  of  having  more  than 
one  alternative  fall  between  criteria.  Figure  12a  indicates  that  as  Number  of  Alternatives  increased, 
exhaustive  search  proportions  decreased  and  redundant  search  proportions  increased.  Thus, 
the  decreases  in  Areas  B  and  E  under  the  probability  cur'/es  in  Figure  10  that  result  from  the 
adjustments  in  criteria  are  exceeded  by  overall  probability  gains  that  result  from  increases  in 
the  Number  of  Alternatives  competing  for  selection. 


Effects  of  Variations  in  Omi ssion  Probability 

Mode!  Parameter  Relationships.  Because  all  the  results  listed  in  Figures  12a  and  b  were 
computed  for  trials  where  the  Correct  Alternative  was  Present.  Omission  Probability  could  affect 
menu  task  performance  only  as  med'ated  through  the  effect  it  had  on  the  two  model  criteria. 
As  indicated  in  Figure  12b,  Omission  Probability  significantly  affected  variations  in  the  rapid 
rejection  criterion,  CO,  but  did  not  have  a  significant  effect  on  the  Cl  rapid  selection  criterion. 
These  results  indicate  that  as  Omission  Probability  increased,  a  stronger  semantic  relationship 
between  target  and  alternatives  was  needed  to  satisfy  requirements  of  the  00  criterion. 
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Effects  on  Response  Accuracy.  A  significant  effect  due  to  variations  in  Omission  Probability 
was  obtained  for  the  proportions  of  response  outcomes  resulting  in  hits.  As  Omission  Probability 
increased,  proportions  of  hits  decreased.  This  effect  is  explained  in  terms  of  the  model  by  the 
effect  Omission  Probabiiity  was  shown  to  h.;ve  on  the  CO  rapid  rejection  criterion.  As  Omission 
Probability  increased,  the  semantic  relatedness  requirements  to  meet  this  threshold  increased. 
As  a  result,  the  probability  of  the  correct  alternative  having  a  semantic  distance  value  that  could 
not  meet  the  CO  criterion  increased  with  increasing  Omission  Probability.  This  resulted  in  an 
increase  in  the  likelihood  of  immediately  rejecting  the  correct  alternative  as  a  response  candidate. 
Thus,  tile  probability  of  a  hit  decreased  at  the  expense  of  an  increase  in  the  probability  of 
omission  misses. 

Eiiects  on  Search  Strategies.  Because  CO  was  the  only  criterion  that  was  influenced 
significantly  by  variations  in  Omission  Probability,  the  model  predicts  that  the  only  types  of 
searches  that  should  be  affected  by  variations  in  Omission  Probability  are  redundant  and 
exhaustive.  More  specifically,  due  to  the  fact  that  semantic  relatedness  requirements  to  meet 
the  CO  criterion  significantly  increased  as  Omission  Probability  increased,  the  model  predicts 
that  the  probability  of  having  more  than  one  alternative  fall  between  criteria  decreases.  Thus, 
exhaustive  searches  are  predicted  to  increase  at  the  expense  of  decreases  in  redundant  searches. 
Further,  because  the  partial  coefficient  for  the  Omission  Probability  predictor  was  not  significant 
in  the  regression  on  C1  estimates,  the  probability  that  at  least  one  alternative  will  satisfy  the 
rapid  selection  criterion  should  remain  relatively  stable  across  varying  levels  of  Omission  Probability. 
Therefore,  self-terminating  search  proportions  are  predicted  to  remain  relatively  constant  across 
varying  levels  of  Omission  Probability. 

The  results  of  search  strategy  regressions  summarized  in  Figure  12a  indicate  that  redundant 
searches  decreased  as  Omission  Probability  increased,  as  predicted  by  the  model.  The  results 
obtained  for  self-terminating  and  exhaustive  searches,  however,  were  contrary  to  model  predictions. 
Omission  Probability  had  no  significant  effect  on  proportions  of  exhaustive  searches.  Further, 
self-terminating  searches  increased  as  Omission  Probability  increased. 

It  is  evident  that  the  relationships  between  Omission  Probability  and  search  strategies  indicated 
through  the  analysis  of  performance  data  are  inconsistent  with  processes  suggested  by  the 
model.  Analyses  of  performance  data  suggest  that  as  Omission  Probability  increased,  the 
relatedness  requirements  associated  with  the  Cl  criterion  decreased.  This  decrease  in  Cl 
requirements,  combined  with  the  increase  in  CO  requirements  that  was  indicated  through  the 
regression  on  model-fitting  data,  suggests  a  process  that  is  consistent  with  all  obtained  performanc  i 
results.  Conceptually,  this  process  proposes  that  subjects  required  greater  relatedness  of 
alternatives  for  selection  consideration;  however,  once  the  relatedness  exceeded  this  threshold, 
subjects  required  lower  relatedness  of  alternatives  for  immediate  selection.  Exactly  why  the 
results  from  the  data-fitting  regressions  on  the  Cl  criterion  were  contrary  to  processes  suggested 
by  the  analysis  of  search  strategy  performances  cannot  be  ascertained  at  this  time.  It  is 
suspected  that  differences  in  the  units  of  analysis  and  resulting  differences  in  the  power  of  the 
two  sets  of  analyses  may  have  contributed  to  this  inconsistency  of  results. 


Effects  of  Variations  in  Dc 


Model  Parameter  Relationships.  Method  A  procedures  for  deriving  criteria  values  were  based 
on  estimates  of  Dc  and  Di  distribution  parameters  computed  across  sets  of  trials  as  defined  by 
the  cells  of  the  experimental  design.  Thus,  the  partial  coefficients  for  the  Dc  and  Di  parameters 
summarized  in  Figure  12b  illustrate  the  relationships  between  variations  in  model  criteria  and 
distribution  parameters  across  sets  of  trials  where  Alternatives  and  Omission  Probability  within 
each  set  were  held  constant.  The  performance  regression  results  listed  in  Figure  12a  were 
based  on  performances  for  individual  trials  across  all  cells  of  the  design.  Whereas  Number  of 
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Alternatives  and  Omission  Probability  were  held  constant  for  trials  within  each  cell,  Dc  and  Di 
values  varied  from  trial  to  trial.  Therefore,  the  use  of  model  criteria  regression  results  in 
explaining  the  effects  that  variations  in  Dc  and  Di  had  on  menu  task  performances  is  complicated 
by  the  use  of  different  units  of  analysis  for  these  two  sets  of  regressions.  In  addition,  the 
relationship  between  variations  in  the  standard  deviation  of  distance  values  across  sets  of  trials 
and  variations  in  the  actual  distance  values  from  trial  to  trial  is  unknown.  For  these  reasons, 
it  is  difficult  to  map  the  effects  on  criteria  that  were  due  to  differences  in  the  standard  deviations 
of  the  two  model  distributions  onto  the  effects  that  distance  variations  had  on  menu  task 
performances.  These  problems  notwithstanding,  the  explanation  of  the  Dc  and  Di  effects  on 
task  performance  will  be  discussed  by  initially  considering  model  predictions  for  the  case  where 
Dc  and  Di  vary  across  relatively  stable  criteria,  and  secondly,  by  considering  model  predictions 
for  variations  in  distance  means  across  sets  of  trials. 

Considering  the  criteria  regression  results  of  the  analyses  for  the  Dc  parameters  in  Figure  12b, 
significant  partial  coefficients  were  obtained  for  variations  in  Dc  distribution  means  across  sets 
of  trials  in  the  regression  on  the  CO  criterion.  However,  partial  coefficients  for  Dc  parameters 
were  not  significant  in  regressions  on  the  Cl  criterion.  These  results  indicate  that  as  Dc  means 
increased  across  sets  of  trials,  the  only  significant  change  observed  for  values  of  model  criteria 
was  a  decrease  in  semantic  relatedness  requirements  associated  with  the  CO  criterion. 

Effects  on  Response  Accuracy.  On  any  given  trial,  as  the  Dc  value  increases,  the  lower 
the  probability  will  be  that  the  correct  alternative  will  have  a  semantic  similarity  to  the  target 
that  exceeds  the  similarity  between  incorrect  alternatives  and  the  target.  Thus,  for  stable  criteria, 
increases  in  Dc  are  predicted  to  decrease  proportions  of  response  outcomes  that  result  in  a 
hit.  Across  sets  of  trials,  as  the  Dc  means  increased,  relatedness  requirements  associated  with 
the  CO  criterion  significantly  decreased.  While  allowing  correct  alternatives  with  a  greater  range 
of  Dc  values  to  be  considered  for  selection,  the  change  in  CO  requirements  does  nothing  to 
offset  the  decreased  level  of  selection  competition  produced  by  the  correct  alternative.  Thus, 
decreases  in  hit  probability  are  also  predicted  for  increases  in  Dc  means  across  sets  of  trials. 
As  indicated  in  Figure  12a,  regression  results  obtained  for  hit  performances  are  consistent  with 
these  predictions.  Across  all  trials,  as  Dc  values  increased,  the  proportion  of  hits  decreased. 

Effects  on  Search  Strategy.  For  any  given  trial,  as  Dc  increases,  the  probability  that  the 
relatedness  of  the  correct  alternative  will  exceed  relatedness  requirements  set  by  the  rapid 
selection  criterion  decreases.  Thus,  for  stable  criteria,  increases  in  Dc  are  predicted  to  decrease 
proportions  of  self-terminating  searches.  As  indicated  in  Figure  12b,  increases  in  Dc  means 
were  shown  to  decrease  relatedness  requirements  set  by  the  rapid  rejection  criterion  (as  indicated 
by  increases  in  values  of  CO).  This  has  the  effect  of  incieasing  the  number  of  alternatives  that 
fall  between  criteria.  The  combination  of  these  events  results  in  predicted  increases  in  proportions 
of  both  exhaustive  and  redundant  searches  at  the  expense  of  decreases  in  self-terminating 
searches.  The  search  strategy  regression  results  indicated  in  Figure  12a  were  consistent  with 
these  predictions.  Across  all  trials,  as  Dc  averages  increased,  self-terminating  searches  decreased 
and  proportions  of  exhaustive  and  redundant  searches  increased. 


Effects  of  Variations  in  Di 


Model  Parameter  Relationships.  As  shown  in  Figure  12b,  the  effect  of  variations  in  Di  means 
across  sets  of  trials  was  significant  for  the  regression  on  the  CO  criterion,  but  was  not  significant 
for  the  Cl  regression.  As  the  Di  means  increased,  the  semantic  relatedness  requirements 
associated  with  the  CO  criterion  decreased.  Conversely,  the  effect  of  variations  in  the  Di  standard 
deviation  across  sets  of  trials  was  not  significant  for  the  CO  regression,  but  its  effect  was 
significant  for  the  regression  on  the  Cl  criterion.  As  noted  during  discussion  of  the  effects  of 
variations  in  Dc,  the  comparison  between  the  effect  that  varicK''ns  in  Di  had  on  menu  task 
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performance  and  the  effect  that  Di  had  on  model  criteria  is  complicated  by  the  fact  that  the 
two  sets  of  regressions  were  based  on  different  units  of  analysis.  The  explanation  of  performance 
results  for  variations  in  Di,  therefore,  will  follow  the  same  theme  that  was  used  to  discuss  the 
effects  of  variations  in  Dc  Predicted  effects  of  variations  in  Di  initially  will  be  considered  for 
the  case  where  Di  varies  across  relatively  stable  criteria.  Secondly,  model  predictions  will  be 
derived  for  variations  in  Di  means  across  sets  of  triais. 

Effects  ofi  Respons  i  Accuracy  For  any  given  trial,  as  Di  increases,  the  probability  that  the 
relatedness  of  any  one  of  the  incorrect  alternatives  in  the  menu  will  exceed  the  relatedness  of 
the  correct  alternative  decreases.  Thus,  for  stable  criteria,  increases  in  Di  are  predicted  to 
increase  the  proportion  of  response  outcomes  resulting  in  hits.  Across  sets  of  trials,  as  the  Di 
means  increase,  the  relatedness  requirements  of  the  CO  criterion  decrease.  Although  this  results 
in  allowing  incorrect  alternatives  with  a  greater  range  of  Di  values  to  be  considered  for  selection, 
the  change  in  CO  requirements  does  nothing  to  offset  the  decreased  level  of  selection  competition 
produced  by  incorrect  alternatives.  Thus,  increases  in  hit  proportions  are  also  predicteo  for 
increases  in  Di  means  across  sets  of  trials.  The  results  obtained  for  the  regression  on  proportion 
of  hit  responses  shown  in  Figure  12a  are  consistent  with  model  predictions.  Across  all  trials, 
the  proportions  of  hits  increased  as  a  function  of  increases  in  Di  averages. 

Search  Strategy  Performances.  As  shown  in  Figure  12a,  the  only  significant  partial  coefficient 
obtained  for  the  Di  predictor  was  in  the  regression  on  redundant  search  proportions.  As  Di 
average  distances  increased,  the  proportion  of  redundant  searches  decreased.  Examination  of 
Figure  10  indicates  that  incorrect  alternatives  having  high  Di  values  are  predicted  to  have  a  low 
probability  of  satisfying  relatedness  requirements  associated  with  the  CO  criterion.  Thus,  alternatives 
having  high  Di  values  are  predicted  to  be  immediately  rejected  The  greater  the  probability  of 
immediately  rejecting  incorrect  menu  alternatives,  the  tower  is  the  probability  that  more  than 
one  alternative  in  the  menu  will  be  considered  for  selection.  The  effect  of  this  process  is  offset 
to  some  extent  for  increases  in  Di  means  across  sets  of  trials.  As  indicated  in  Figure  12b,  the 
relatedness  requirements  of  the  CO  criterion  decrease  with  increasing  Di  means.  Menu  task 
performance  results,  however,  suggest  that  the  decrease  in  CO  relatedness  requirements  is  not 
sufficient  to  counter  the  increased  number  of  incorrect  alternatives  that  are  immediately  rejected 
when  Di  values  are  high  This  process  results  in  the  obtained  effect  that  as  Di  averages 
increase,  redundant  search  proportions  significantly  decrease 


Summary 

Analysis  of  menu  task  performance  data  showed  that  menu  design  factors  significantly 
influence  both  search  strategy  and  response  accuracy.  The  two-criterion  menu  model  provided 
a  theoretical  framework  of  menu  search  and  decision  processes  that  was  employed  to  explain 
variations  in  menu  task  performances.  Through  the  fitting  of  menu  task  data  to  prediction 
equations  derived  for  the  menu  model,  the  relationship  between  model  criteria  and  menu  design 
parameters  was  assessed  The  discussions  presented  in  this  section  demonstrated  how  the 
processes  suggested  by  the  model  accommodate  most  of  the  findings  derived  from  analysis  of 
menu  task  performance  data  An  exception  to  the  consistency  between  model  predictions  and 
menu  task  performance  was  noted  for  the  relationship  between  variations  in  Omission  Probability 
and  proportions  of  search  strategies  employed  by  subjects  Though  the  model  predicted  the 
results  obtained  for  Omission  Probability  effects  on  redundant  searches,  the  Omission  Probability 
effects  that  were  indicated  for  self-terminating  and  exhaustive  searches  were  contrary  to  model 
predictions.  This  fact  notwithstanding,  performance  results  obtained  for  all  other  menu  design 
factors  studied  in  this  investigation  were  consistent  with  processing  suggested  by  the  model. 

Although  the  results  of  the  present  investigation  contribute  to  the  understanding  of  menu 
search  and  decision  processes,  several  issues  remain  unresolved.  What  follows  is  a  summary 
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of  the  contribution  of  the  present  work  to  the  problem  of  modeling  requirements  for  optimal 
menu  design.  Issues  to  be  resolved  through  further  study  are  discussed,  and  a  research  agenda 
is  proposed  for  the  development  of  an  operational  model  that  will  provide  menu  designers  the 
information  necessary  to  optimize  specific  menu  designs. 


Research  Agenda 

Successful  implementation  of  menu-driven  systems  across  a  variety  of  conditions  requires  a 
model  in  which  all  factors  affecting  search  strategy  and  response  accuracy  are  included.  The 
results  of  the  present  investigation  indicate  the  necessity  of  including  Number  of  Alternatives, 
Omission  Probability,  and  semantic  relatedness  between  target  and  menu  alternatives  in  such  a 
model.  Processes  proposed  under  the  two-criterion  menu  model  meet  this  requirement,  and 
were  shown  to  accommodate  many  of  the  findings  obtained  from  analysis  of  menu  task 
performance  data.  Although  the  results  of  the  model-fitting  procedures  described  in  Section  V 
are  relevant  to  the  understanding  of  menu  search  and  decision  processes,  the  procedures  fall 
short  of  a  totally  rigorous  test  of  the  model.  The  testing  issue  is  left  to  future  research.  In 
addition,  refinement  of  the  model  for  use  as  an  operational  tool  to  resolve  menu  design  issues 
for  specific  applications  would  also  require  further  work. 

Such  effort  might  begin  by  further  studying  the  problem  of  deriving  a  priori  estimates  of 
model  criteria.  Estimation  of  reliable  menu  criteria  values  through  the  linear  combination  of 
menu  design  parameters  certainly  appears  feasible  following  from  the  results  of  the  model-fitting 
analyses  However,  the  stability  of  model  criteria  relationships  to  menu  design  parameters  across 
a  variety  of  menu  task  situations  needs  to  be  determined.  Further  work  is  also  required 
concerning  the  optimal  manner  by  which  model  distribution  parameters  might  best  be  estimated. 
Finally,  tests  designed  to  assess  the  predictive  value  of  the  model  for  several  different  types  of 
menu  stimuli  need  to  be  conducted  to  validate  the  accuracy  of  the  model. 

Once  the  model  and  its  assumptions  have  been  validated  (and/or  modified),  the  logical 
conclusion  of  this  work  would  be  to  refine  the  model  into  an  operational  tool  to  be  used  for 
optimization  of  specific  computer  menu  designs.  This  goal  could  be  achieved  through  extending 
the  model’s  predictive  capabilities  from  its  current  focus,  which  concerns  partial  search  menu 
task  performances,  to  the  general  activity  of  navigation  through  menu  hierarchies.  This  would 
require  consideration  of  breadth/depth  performance  tradeoffs  through  computation  of  model 
parameter  variations,  and  assessment  of  resulting  variations  in  model  performance  predictions. 

Several  implications  were  derived  from  the  present  effort  with  respect  to  the  significance  of 
menu  design  factors  on  task  performance,  and  with  respect  to  the  modeling  of  search  and 
decision  processes  involved  during  the  performance  of  the  partial  search  menu  task.  Results 
demonstrated  the  inadequacy  of  menu  optimization  models  which  are  based  solely  on  subsets 
of  the  design  factors  studied  here.  Both  the  literature  and  the  results  of  the  present  experiment 
suggest  a  complex  processing  phenomenon  involved  in  menu  search  and  decision  making  which 
involves  all  factors  studied.  Though  falling  short  of  a  definitive  test,  the  procedures  used  in 
the  present  investigation  demonstrated  the  two-criterion  menu  model  to  be  capable  of  fitting 
empirically  obtained  performance  data  in  a  manner  that  contributes  to  the  understanding  of  the 
relationships  between  menu  design  factors  and  task  performance. 


VII.  CONCLUSIONS 

Results  of  the  present  investigation  showed  the  significant  effects  that  Number  of  Alternatives, 
Omission  Probability  and  semantic  relationships  among  menu  stimuli  had  on  menu  task 
performance.  It  was  also  shown  how  variations  in  Number  of  Alternatives  across  levels  of  a 
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menu  hierarchy  were  confounded  by  variations  in  Dc  and  Di  distances.  These  results  indicated 
the  complexity  of  the  relationship  that  exists  between  variations  in  these  factors  and  search 
strategy  and  response  outcome  performances.  These  findings  suggested  that  in  order  to  best 
understand  this  complex  relationship,  a  framework  that  accounts  for  the  effects  of  all  these 
factors  is  required.  Such  a  framework  was  developed  through  the  proposed  two-criterion  menu 
model.  The  model  was  based  on  the  relationship  that  exists  between  the  target  and  menu 
alternatives  for  menus  of  any  size  and  on  the  probability  that  the  Correct  Alternative  is  Present 
in  the  menu.  Although  it  was  shown  that  processes  proposed  under  this  model  could  be  used 
to  account  for  many  of  the  findings  obtained  from  analysis  of  performance  data,  these  procedures 
fell  short  of  a  definitive  test  of  the  model’s  validity.  Still,  there  are  several  implications  that  can 
be  drawn  from  this  work  and  used  to  form  general  recommendations  for  optimizing  the  structure 
of  computer  menus. 

To  optimize  the  menu  structure,  the  designer  must  consider  the  effects  that  each  menu  in 
the  hierarchy  has  on  response  speed  and  accuracy.  As  was  indicated  in  the  Section  II  review 
of  the  literature,  the  sum  of  these  effects  across  all  levels  of  the  hierarchy  has  traditionally  been 
used  as  the  basic  unit  of  measurement  for  determining  optimum  menu  design.  Optimal  structure 
has  been  discussed  typically  in  terms  of  designs  in  which  response  time  is  minimized  or  response 
accuracy  is  maximized,  or  in  terms  of  tradeoffs  between  response  speed  and  accuracy  across 
all  levels  of  the  menu  structure. 

With  regard  to  menu  optimization  studies,  a  primary  issue  has  been  the  Number  of  Alternatives 
that  should  be  incorporated  in  the  design  of  the  menu  structure.  The  present  investigation 
studied  this  issue  in  terms  of  factor  effects  on  performance  for  individual  menus  that  were 
derived  from  various  levels  of  a  menu  structure.  The  results  showed  that  Number  of  Alternatives 
was  not  only  a  significant  factor  in  terms  of  its  partial  correlation  to  response  accuracy,  but 
also  significantly  confounded  by  variations  in  the  relationship  between  correct  and  incorrect 
alternatives  (i.e.,  Di  values).  In  addition,  the  strength  of  the  relationship  between  Number  of 
Alternatives  and  Di  variations  was  also  shown  to  vary  across  levels  in  the  menu  hierarchy. 

The  implication  these  findings  have  with  respect  to  optimum  Number  of  Alternatives  is  that 
it  is  highly  probable  that  this  number  will  vary  across  menu  levels  for  any  given  hierarchy. 
Because  increases  in  Number  of  Alternatives  differentially  affect  Di-NN  and  Di-AVE  values  for 
varying  hierarchy  levels,  the  tradeoffs  among  the  influences  each  of  these  factors  has  on  task 
performance  do  not  appear  to  be  consistent  across  levels  in  the  hierarchy.  In  terms  of  the 
two-criterion  menu  model,  this  process  is  further  complicated  by  variations  both  in  Dc  distributions 
and  in  criteria  employed  by  subjects  across  levels  of  the  menu  structure.  Thus,  while  the 
relationships  among  menu  alternatives  are  dependent  both  on  the  hierarchical  level  from  which 
alternatives  are  derived  and  on  the  Number  of  Alternatives  contained  in  the  menu,  the  overall 
performance  effect  of  variations  in  Number  of  Alternatives  is  also  dependent  on  selection 
thresholds.  It  is  concluded  that  the  optimum  Number  of  Alternatives  is  dependent  on  the 
relationships  among  all  model  parameters  for  specific  levels  in  the  hierarchy  from  which  menus 
are  constructed. 

In  summary,  though  the  development  of  an  operational  model  that  could  be  applied  to  the 
optimization  of  specific  menu  applications  requires  additional  effort,  the  two-criterion  model 
developed  during  this  investigation  provides  both  a  quantitative  and  theoretical  framework  to 
guide  the  continuation  of  this  work.  Additionally,  results  of  the  experiment  reported  in  this  report 
indicate  th®  complexity  of  the  relationships  among  menu  design  factors  and  of  their  effects  on 
task  performance.  Although  work  reported  in  the  human  factors  literatuie  has  searched  for  an 
optimum  Number  of  Alternatives  that  could  be  used  to  guide  menu  designers,  results  of  this 
work  appear  to  be  confounded  by  the  failure  to  consider  the  effects  that  Number  of  Alternatives 
has  on  semantic  relationships  among  menu  stimuli. 
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Results  of  the  present  investigation  suggest  that  the  optimum  menu  structure  may  be  one 
in  which  the  Number  of  Alternatives  varies  as  the  user  travels  through  the  menu  hierarchy.  The 
optimum  Number  of  Alternatives  for  a  given  level  of  the  hierarchy  is  not  only  expected  to  vary 
from  one  level  to  the  next  but,  due  to  varying  relationships  among  menu  design  factors,  also 
expected  to  vary  from  one  menu  application  to  another.  This  optimum  is  dependent  on  the 
effects  that  ail  model  parameters  have  on  task  performance  at  each  level  of  the  menu  structure. 
Thus,  in  deciding  upon  the  Number  of  Alternatives  to  use  for  a  specific  level  of  the  hierarchy, 
the  menu  designer  must  consider  the  effect  this  number  will  have  on  menu  breadth/depth,  model 
criteria,  and  the  relationships  among  targets  and  menu  alternatives. 
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APPENDIX:  MODEL  PARAMETER,  RMSE,  AND  CORRELATION  VALUES 
FOR  METHOD  A  AND  B  BEST-FIT  SOLUTIONS 
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Table  A-1.  Method  A  Model  Solutions  for  7  Correct  Alternative  Present  Equations 


Hierarchy 

-  Level  III 

Omissn  8  alts 

4  alts  (Nesting 

4  alts  (Nesting 

2  aits 

Prob  (menu  set  1) 

close  -  set  2) 

dist  -  set  3) 

(menu  set  4) 

M1 

=  1.385 

Ml 

=  1.320 

Ml 

1.442 

Ml 

=  1.333 

SD1 

=  .319 

SD1 

=  .337 

SD1 

= 

.283 

SD1 

=  .413 

0%  M2 

=  .845 

M2 

=  .900 

M2 

= 

.858 

M2 

=  .839 

SD2 

=  .280 

SD2 

=  .264 

SD2 

= 

.322 

SD2 

=  .288 

CO 

=  1.036 

CO 

=  1.033 

CO 

= 

1.121 

CO 

=  1.133 

Cl 

=  .658 

Cl 

=  .436 

Cl 

= 

.552 

Cl 

=  .516 

RMSE 

=  .080 

RMSE 

=  .150 

RMSE 

.082 

RMSE 

=  .107 

Ml 

=  1.385 

Ml 

=  1.335 

Ml 

= 

1.470 

Ml 

=  1.390 

SD1 

=  .319 

SD1 

=  .334 

SD1 

.277 

SD1 

=  .411 

12.5%  M2 

=  .845 

M2 

=  .933 

M2 

.894 

M2 

=  .871 

SD2 

=  .280 

SD2 

=  .256 

SD2 

= 

.323 

SD2 

=  .286 

CO 

=  .935 

CO 

=  1.010 

CO 

= 

1.104 

CO 

=  1.098 

Cl 

=  .572 

Cl 

=  .418 

Cl 

= 

.471 

Cl 

=  .474 

RMSE 

=  .105 

RMSE 

=  .131 

RMSE 

= 

.070 

RMSE 

=  .045 

Ml 

= 

1.385 

Ml 

= 

1.340 

Ml 

= 

1.469 

Ml 

= 

1.397 

SD1 

= 

.319 

SOI 

= 

.336 

SDI 

= 

.283 

SDI 

= 

.419 

25%  M2 

= 

.845 

M2 

= 

.910 

M2 

= 

.910 

M2 

= 

.904 

SD2 

= 

.280 

SD2 

= 

.242 

SD2 

= 

.334 

SD2 

= 

.280 

CO 

= 

.968 

CO 

= 

.991 

CO 

= 

1.107 

CO 

= 

1.105 

Cl 

= 

.695 

Cl 

= 

.420 

Cl 

= 

.557 

Cl 

= 

-.202 

RMSE 

= 

.082 

RMSE 

= 

.086 

RMSE 

=: 

.058 

RMSE 

= 

.067 

Ml 

— 

1.385 

Ml 

_ 

1.326 

Ml 

1.464 

Ml 

_ 

1.413 

SD1 

= 

.319 

SDI 

= 

.337 

SDI 

= 

.282 

SDI 

.413 

37.5%  M2 

= 

.845 

M2 

= 

.905 

M2 

= 

.902 

M2 

= 

.923 

SD2 

= 

.280 

SD2 

= 

.265 

SD2 

= 

.280 

SD2 

= 

.303 

CO 

= 

.916 

CO 

= 

.988 

CO 

= 

1.069 

CO 

= 

1.109 

Cl 

= 

.711 

Cl 

= 

.627 

Cl 

= 

.729 

Cl 

= 

-.496 

RMSE 

= 

.055 

RMSE 

= 

.049 

RMSE 

= 

.012 

RMSE 

= 

.046 

Table  A-1.  (Concluded) 


Hierarchy  -  Level  V 

8  aits  4  alts  (Nesting  4  alts  (Nesting  2  alts 

(menu  set  5)  close  -  set  6)  dist  -  set  7)  (menu  set  8) 


M1 

=  1.295 

Ml  = 

.976 

Ml 

=  1.770 

Ml  = 

.797 

SD1 

=  .449 

SD1  = 

.351 

SD1 

=  .379 

SD1  = 

.356 

M2 

=  .666 

M2  = 

.611 

M2 

=  .680 

M2  = 

.545 

SD2 

=  .252 

SD2  = 

.171 

SD2 

=  .255 

SD2  = 

.180 

CO 

=  .848 

CO  = 

.748 

CO 

=  1.137 

CO  = 

.745 

Cl 

=  .536 

Cl  = 

.485 

Cl 

=  .508 

Cl  = 

.447 

RMSE 

=  .101 

RMSE  = 

.109 

RMSE 

=  .016 

RMSE  = 

.091 

Ml 

=  1.315 

Ml 

=  1.011 

Ml 

=  1.809 

Ml  = 

.847 

SD1 

=  .437 

SD1 

=  .348 

SD-: 

=  .371 

SD1  = 

.352 

M2 

=  .731 

M2 

=  .613 

M2 

=  .700 

M2  = 

.532 

SD2 

=  .256 

SD2 

=  .184 

SD2 

=  .266 

SD2  = 

.190 

CO 

=  .811 

CO 

=  .719 

CO 

=  1.030 

CO  = 

.732 

Cl 

=  .528 

Cl 

=  .442 

Cl 

=  .534 

Cl  = 

.387 

RMSE 

=  .093 

RMSE 

=  .113 

RMSE 

=  .009 

RMSE  = 

.083 

Ml 

=: 

1.342 

Ml 

= 

1.008 

Ml 

= 

1.844 

Ml 

= 

.880 

SD1 

.407 

SD1 

ss 

.360 

SD1 

.362 

SD1 

= 

.368 

M2 

= 

.782 

M2 

= 

.629 

M2 

.722 

M2 

= 

.547 

SD2 

.267 

SD2 

= 

.192 

SD2 

= 

.274 

SD2 

= 

.202 

CO 

= 

.904 

CO 

= 

.735 

CO 

= 

1.095 

CO 

= 

.763 

Cl 

ss 

.698 

Cl 

.460 

Cl 

= 

.710 

Cl 

= 

.403 

RMSE 

- 

.043 

RMSE 

= 

.096 

RMSE 

=: 

.008 

RMSE 

= 

.083 

Ml 

_ 

1.403 

Ml 

_ 

1.036 

Ml 

= 

1.850 

Ml 

.935 

SD1 

= 

.424 

SD1 

= 

.371 

SD1 

= 

.369 

SD1 

= 

.378 

M2 

.883 

M2 

= 

.632 

M2 

= 

.754 

M2 

= 

.552 

SD2 

= 

.286 

SD2 

= 

.198 

SD2 

.291 

SD2 

= 

.223 

CO 

= 

.975 

CO 

= 

.768 

CO 

= 

1.134 

CO 

= 

.811 

Cl 

= 

.774 

Cl 

= 

.500 

Cl 

.736 

Cl 

= 

.426 

RMSE 

s= 

.034 

RMSE 

.073 

RMSE 

.010 

RMSE 

= 

.058 

Lm«nd. 


Ml  = 

Oi  distribution  M. 

M2  = 

Dc  distribution  M. 

SD1  = 

Di  distribution  SD. 

SD2  = 

Dc  distribution  SD. 

CO  = 

rapid  rejection 

Cl  = 

rapid  selection 

criterion. 

criterion. 

Nota.  Free  parameters:  CO  and  Cl.  Fixed  parameters:  Number  of  Alternatives,  Omission 
Probability,  Ml,  M2,  ^1,  and  SD2. 
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Table  A-2.  Method  B  Model  Solutions  for  7  Correct  Alternative  Present  Equations 


Omissn 

Prob 

Hierarchy 

-  Level 

III 

8  alts 

(menu  set  1) 

4  alts  (Nesting 
close  -  set  2) 

4  aits  (Nesting 
dist  -  set  3) 

2  alts 

(menu  set  4) 

M1 

=  1.385 

Ml 

=  1.320 

Ml 

— 

1.442 

Ml 

_ 

1.333 

SD1 

=  .130 

SD1 

=  .138 

SD1 

= 

.083 

SD1 

.087 

0% 

M2 

=  .845 

M2 

=  .900 

M2 

= 

.858 

M2 

.839 

SD2 

=  .281 

SD2 

=  .064 

SD2 

=: 

.256 

SD2 

.159 

CO 

=  1.211 

CO 

=  1.127 

CO 

= 

1.319 

CO 

1,205 

Cl 

=  .718 

Cl 

=  .837 

Cl 

= 

.697 

Cl 

.726 

RMSE 

=  .028 

RMSE 

=  .011 

RMSE 

= 

.032 

RMSE 

= 

.005 

Ml 

=  1.385 

Ml 

= 

1.335 

Ml 

=  1.470 

Ml 

=  1.390 

SD1 

=  .191 

SD1 

= 

.120 

SD1 

=  .095 

SD1 

=  .265 

12.5%  M2 

=  .845 

M2 

= 

.933 

M2 

=  .894 

M2 

=  .871 

SD2 

=  .180 

SD2 

= 

.155 

SD2 

=  .325 

SD2 

=  .186 

CO 

=  1.025 

CO 

= 

1.117 

CO 

1.294 

CO 

=  1.055 

Cl 

=  .728 

Cl 

= 

.764 

Cl 

=  .610 

Cl 

=  .667 

RMSE 

=  .010 

RMSE 

= 

.021 

RMSE 

=  .025 

RMSE 

=  .015 

Ml 

z= 

1.385 

Ml 

= 

1.340 

Ml 

= 

1.469 

Ml 

= 

1.397 

SD1 

= 

.131 

SD1 

= 

.177 

SD1 

= 

.106 

SD1 

.158 

25%  M2 

= 

,845 

M2 

= 

.910 

M2 

= 

.910 

M2 

= 

.904 

SD2 

= 

.315 

SD2 

= 

.142 

SD2 

= 

.342 

SD2 

= 

.180 

CO 

= 

1.159 

CO 

= 

1.041 

CO 

= 

1.275 

CO 

= 

1.085 

Cl 

= 

.745 

Cl 

= 

.737 

Cl 

= 

.659 

Cl 

= 

.615 

RMSE 

= 

.014 

RMSE 

= 

.019 

RMSE 

= 

.026 

RMSE 

.012 

Ml 

— 

1.385 

Ml 

1.326 

Ml 

1.464 

Ml 

1.413 

SD1 

= 

.219 

SD1 

= 

.164 

SD1 

— 

.196 

SD1 

= 

.192 

37.5%  M2 

- 

.345 

M2 

= 

.905 

M2 

.902 

M2 

= 

.923 

SD2 

= 

.302 

SD2 

.272 

SD2 

= 

.366 

SD2 

= 

.203 

CO 

= 

.996 

CO 

= 

1.086 

CO 

= 

1.145 

CO 

= 

1.092 

Cl 

= 

.749 

Cl 

= 

.696 

Cl 

= 

.700 

Cl 

.583 

RMSE 

= 

.017 

RMSE 

= 

.012 

RMSE 

= 

.009 

RMSE 

= 

.011 
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Table  A-2.  (Concluded) 


Hierarchy 

-  Level 

V 

8  alts 

(menu  set  5) 

4  alts  (Nesting 
close  -  set  6) 

4  alts  (Nesting 
dist  -  set  7) 

2 

(menu 

alts 
set  8) 

Ml 

_ 

1.295 

Ml 

=  .976 

Ml 

- 

1.770 

Ml 

.797 

SD1 

= 

.307 

SD1 

=  .174 

SDI 

= 

.279 

SDI 

= 

.177 

M2 

= 

.666 

M2 

=  .611 

M2 

= 

.680 

M2 

= 

.545 

SD2 

= 

.124 

SD2 

=  .100 

SD2 

= 

.252 

SD2 

= 

.047 

CO 

= 

.900 

CO 

=  .770 

CO 

= 

1.276 

CO 

= 

.634 

Cl 

.644 

Cl 

=  .573 

Cl 

= 

.525 

Cl 

= 

.530 

RMSE 

= 

.022 

RMSE 

=  .030 

RMSE 

= 

.006 

RMSE 

= 

.026 

M1 

1.315 

Ml 

=  1.011 

Ml 

_ 

1.809 

Ml 

.847 

SD1 

= 

.276 

SD1 

=  .086 

SDI 

= 

.354 

SDI 

= 

.040 

M2 

.731 

M2 

=  .613 

M2 

= 

.700 

M2 

= 

.532 

SD2 

= 

.090 

SD2 

=  .184 

SD2 

= 

.266 

SD2 

= 

.152 

CO 

= 

.837 

CO 

=  .853 

CO 

= 

1.031 

CO 

= 

.789 

Cl 

= 

.698 

Cl 

=  .524 

Cl 

s: 

.537 

Cl 

= 

.463 

RMSE 

.018 

RMSE 

=  .020 

RMSE 

= 

.009 

RMSE 

= 

.029 

Ml 

1.342 

Ml 

=  1.008 

Ml 

1.844 

Ml 

.880 

SD1 

= 

.288 

SD1 

=  .176 

SD1 

= 

.347 

SDI 

= 

.113 

M2 

.782 

M2 

=  .629 

M2 

s 

.722 

M2 

= 

.547 

SD2 

= 

.202 

SD2 

=  .081 

SD2 

= 

.374 

SD2 

= 

.064 

CO 

S 

.959 

CO 

=  .737 

CO 

= 

1.225 

CO 

= 

.680 

Cl 

= 

.789 

Cl 

=  .592 

Cl 

=: 

.694 

Cl 

-.522 

RMSE 

= 

.010 

RMSE 

=  .026 

RMSE 

= 

,003 

RMSE 

.011 

Ml 

_ 

1.403 

Ml 

=  1.036 

Ml 

_ 

1.850 

Ml 

_ 

.935 

SD1 

= 

.255 

SD1 

=  .156 

SDI 

= 

.280 

SDI 

= 

.062 

M2 

= 

.883 

M2 

=  .632 

M2 

= 

.754 

M2 

= 

.552 

SD2 

= 

.110 

SD2 

=  .141 

SD2 

= 

,491 

SD2 

= 

.136 

CO 

= 

1.011 

CO 

=  .823 

CO 

= 

1.376 

CO 

= 

.849 

Cl 

.906 

Cl 

=  .588 

Cl 

= 

.705 

Cl 

= 

.511 

RMSE 

.004 

RMSE 

=  .008 

RMSE 

— 

.006 

RMSE 

=: 

.007 

Legend.  Ml 

=  Di  distribution  M, 

M2 

=  Dc 

distribution 

M. 

SD1 

=  Di  distribution  SO. 

SD2 

=  Dc 

distribution 

SD. 

CO 

=  rapid  rejection 

Cl 

=  rapid  selection 

criterion 

criterion. 

Note. 

Free  parameters;  CO, 

C1,  SDI,  and 

SD2.  Fixed 

parameters. 

Number 

of 

Alternatives, 

Omission 

Probability, 

Ml  and  M2. 
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Table  A-3.  Method  A  Summary  Statistics  for  Predicted  Versus  Observed  Proportions 
Across  32  Cells-7  Correct  Alternative  Present  Equation  Solutions 
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Table  A-4.  Method  B  Summary  Statistics  for  Predicted  Versus  Observed 
Proportions  Across  32  Cells--?  Correct  Alternative  Present  Equation  Solutions 


